You can get up Giant's Castle Pass easily in a day from Giant's camp. It's an average pass with a fairly good trail most of the way.
Lotheni Pass has 2 distinct forks, the North Fork is the one indicated on the maps and is definitely a rock pass. The South Fork (the one the cave is in) is a smuggling route complete with a track, and is a much more obvious (and easier) pass than the North Fork (why that fork even got the official title is a mystery to me). The trail contours along the Ka Pitoli ridge and eventually flips over into the next valley in the vicinity of Killer Ridge - that's as far as I've followed it. From there it possibly follows Elandshoek valley out, or it heads on towards Taylor's Pass / The Sisters area where is might link up with yet another smuggling pass which starts near Giant's Castle summit.
Your day 2 would be to descend Lotheni Pass and get as far as possible towards the junction near Taylor's Pass / eChibini Tarn. You might be able to follow smuggling tracks for most, if not all of that. On the other hand you might also face navigational challenges and a bit of bundu-bashing. Day 3 would be relatively easy once you've rejoined the highways of Giants Castle area.
So, while it sounds like could pull it off in 3 days, consider 4 as an extra, also to enjoy it more, and even to climb Giant's Castle, especially if you haven't done it before.
