We went on a day after touring city and getting rained on. So pretty casual (shorts/nice shirts) and it was fine. Was mixture of casual and business attire
People weren't dressed up. Dark jeans and a nice top are fine.
I would agree with Howeirds. I saw someone dressed down, but they looked out of place. No tie is expected for men, but a nice shirt and pants would be best. For women, something similar.
I was advised by my Charlestonian friend to wear a decent pair of trousers and a jacket. There were many in the restaurant more casual but the staff seemed to appreciate our effort in making it a special meal by dressing accordingly. My wife was in a cocktail dress and heels.
Get quick answers from FIG staff and past visitors.