Based on what you have said, I think you will be OK. Yes, there will be nervous moments but I think your enjoyment will far surpass your fear.
I do not like heights and consider myself as having a fear but I still push myself to visit places that involve heights as I don't want to miss out.
For me, the worst parts of the harbour climb is the initial walk out to the ladders. It's a mesh grate and you can see below. The other part was crossing the bridge at the top to get to the other side. Again, it was the traffic below that freaked me out.
I found that looking ahead and up if needed was enough to get me through and end enjoy it.
I have climbed the Eiffel Tower several times now and familiarity does make it easier. The first time I went up I stuck as far away from the edge as I could until I was brave enough to head to the outer rails. Bizarrely, I feel better walking up the ET than taking the lift.
Last year I did the Statue of Liberty and survived. The one place that I did freak out was climbing The Monument in London. When I got to the top, well that was enough for all my fears to come back. My palms are sweaty now even thinking about it.
I still remember years ago when I visited the CNN Tower in Toronto, they had a glass floor. There was no way in h3ll I was going to walk over that even intellectually knowing that I was already up there standing on concrete.