While I absolutely believe the reports of a mugging or two and a police shakedown or two; I firmly believe that's the aberration, not the norm. We certainly have our share of that here in the States.
In my 20+years of going to TJ for mongering, I've never seen a HINT of that kind of trouble. The local politicians and police have a vested interest in not scaring off tourists' money (they'd rather not have the tourists, just their money!)
In the daylight, the border/Adelitas walk is an absolute piece of cake alone. If one were an exceptionally petite, blonde female with a "deer in the headlight" look, who was drunk & devoid of any situational awareness one would STILL probably luck out quite a few times before something happened. That's more than I can say for MOST of the streets I cruise up here.
You don't have to worry about "fitting in" or "standing out". This is a TOURIST area! I've found what others have said to be true: don't flash cash, don't be drunk, obnoxious, or rude, be and look aware of your surroundings and who's where. Sounds pretty much like any city street, huh?
Okay, enough for my ranting & raving. Quit worrying and go for it!
Hawk