Both routes take over 6 hours.
There’s always the 2 days 9 hours walk! I’m guessing that without any stops!
There's also a local bus from Caen to Honfleur (line 20) which costs 5€. If I can take this bus in 19.30, I may get in Honfleur in 21.26.
The only way to cover the remaining 25 km between Honfleur and Le Havre after 21.30 seems to be taxi :(
Is there a decent bike rental service in the region?
Are you making the return journey then? Most bike rentals are return to where you rent from. Plus not likely to be open at that time of night. I’m also assuming you mean pedal cycle. If you went your way to Caen then switched to the train it would work.
Really whilst your question is what is the cheapest surely the reality is what is possible.
As for using a taxi for the last 25km whilst possible I’d be surprised if that was anywhere near fitting the cheap criteria.