How to delete a line in Photoshop
If you draw a straight line in Photoshop and change your mind you can easily delete it by clicking Ctrl+Z/Cmd+Z once to remove the line and a second time to delete the dot.
Alternatively, you can use the marquee tool to select your line and press Ctrl+T/Cmd+T to move the line or adjust the size, length, width or positioning of it.
How to draw stable lines in Photoshop
If you’re looking to reduce shake when you draw but aren’t necessarily wanting to draw straight lines, this answer is for you.
Click the brush tool, select Smoothing and adjust the percentage to smooth out any brushstrokes you make and help lines appear more stable.