Just visiting Big Buddha? You don’t need a driver for that, only a tuktuk or taxi.
If you want a driver, then make it worthwhile and plan a tour of sites in the south that interest you.
In that case, give Big Buddha a miss. The 'temple' itself is nothing to write home about and is, basically, a big money making exercise. Views to Phi Phi can be wonderful, but if it's a hazy day (which is often) you see very little.