How To Handle Pan-handlers?

Sunday we had a great day at Stamp Camp. We made a lot of great projects (which I swear I will post) and we had lots of great food - lots. We had so much food that they ended sending me home with a bunch of food. After Camp I went by the store to grab a big sub and some soup for dinner. When I was leaving I saw a guy, who did not look very old, standing with a sign that said he had lost his home and his job. Now I have seen a lot of these people and I usually feel bad but part of me wonders what is going on, do they really need the help because they have exhausted resources. Now please do not judge me, I know as a Christian I should help everyone I can, but I worked downtown when I was younger and many of the people I saw in these situations wanted to just hang out with their friends and drink and do other things. I also am aware that people do these things because they are caught in a vicious cycle and in many cases it is difficult to break out of it and get a job. That being said I still have a hard time giving people money or anything else. This guy on the other hand really upset me. I thought of all of the food I had in my car and how much I am always shopping. I never carry cash so giving him money was not an option. So the entire way home I was thinking should I have given him something? What would I have given him? I said I was sent home with a lot of food, but I was sent home with large veggie trays, cup cakes and banana bread. So I was thinking he probably did not need the banana bread or the cup cakes because they had so much sugar and would not be satisfying long term also I am sure he had nothing to drink and you can't eat either of those things without something to drink. I thought the veggie tray was a weird option because it was huge and if you don't have basic things like a home and money do you really want to sit somewhere and eat an entire veggie tray? Should I have given him the sub sandwich that I planned to feed my family with? To make matters worse I was full from all the days snacking and nobody ate much of the sub. So here I am on Tuesday still thinking about this guy. I don't know what the right answer is but I did pray for him. I don't mean to sound like a mean heartless person but I am a big believer in teaching a man to fish not giving a man a fish.