It's OK just as long as you don't flip the plug. Watch out for reverse polarity, that will the smoke out....it’s totally ok to hook up a single cell from your battery this way...
My self, I use an old 5V transformer, aka a wall wart, for bench testing. Just cut off the end and solder on an old servo lead. If I accidently flip the plug, nothing happens.