I’d vote for Casa de Fruta because I like it. It’s about 50 miles from San Jose, and travel time will depend a lot on commute traffic. Yes, the San Jose to Gilroy corridor can get ugly on a weekday afternoon. BTW, it has a Hollister address because it has a P.O. box there, but you aren’t actually going down to Hollister; it is on 152, right on your way.
This is an experience, not *just* a motel, so if you want *just* a motel, the next choice is 25 miles farther east on 152, a Motel 6 at the same exit with the Petro Travel Plaza and KOA. If you reach the I-5 interchange, you’ve passed it. (You won’t take I-5 at all, but 99 south).
You will need to pay to drive into the National Park. One fee is good for seven days. Your lodging is about 5 minute drive from the South Entrance, then it's another hour to drive to Yosemite Valley.
I would stay at Casa de Fruta over Los Banos! 😎