Take out the one on the other side and take it to home depot. You can also go to a marine boat store like West Marine and get a stainless steel bolt that won't rust as quick as not galvanized. Be sure to hold the turn signal in and at the position you want it while tightening the bolt back in place.
That's funny - mine were on so dang tight that I ended up stripping the philips screw and had to use a screw extractor to get them out.
They are M6 x 1.00mm thread pitch and 10mm long, btw. Any hardware store with a decent selection of fasteners should have them. I'd recommend going with a socket cap or button head bolt and avoid the philips type screw.