Had An Awkward Moment When Asking Lady Out To Lunch...

This one woman who I've seen living in my neighborhood had caught my attention. She is around 30ish, but you wouldn't know it by her cheerful nature combined with a pristine look when she wears her make-up. Been somewhat friendly casual neighbor for her, and helped her with quite a few things, just trying to be a nice guy, like I do for the rest of my neighborhood...
I'd been meaning to actually ask her out, as I know she isn't seeing people, based on idle chitchat I've shared with her.

I tried to ask if she'd like a lunch or something just the other day, but she seemed to take it the wrong way? She seemed to think I was trying to "hire" her for an afternoon, and she politely declined, saying that she stopped doing "that stuff" long ago... I had to explain to her that it was just a polite invitation to lunch, at a diner just a few blocks down the road, which I couldn't get out the first time I talked due to a slight stutter I have.

Well, she seemed a bit embarrassed afterwards, awkwardly apologizing... She explained that she gets offers still, despite not doing "that work" anymore, because she says people think that a black lady like her is usually a "Ho". She laughed it off as a funny mistake, before saying that she'd think about the lunch offer.

I don't know what to make of it, but it was kind of bold for her to assume such a thing, I feel... I'm not *that* kind of guy, worst I do is browse porn online in some of my freetime when I am not doing my own hobbies, not *hire* any lady for anything of the sort.

Should I be worried or something?