When ghoS is used to mean "go away from", it has -vo':
chaH neH wovmoHlu'be'
chaH 'emvo' ghoS
SuvwI'pu' mangghom yoH
The sun shone not on them only,
Behind them came
An army of brave warriors.
(paq'batlh)
I don't see a "go away from" meaning there. I see a "go/proceed" and I see a context-setting "from behind them", but they don't combine for me the way you seem to be treating it. My interpretation of this text is that ghoS has its typical meaning of "move along a path", with an implied destination here of chaH. Notice the English version of the passage uses the word "came", not "went away from". The phrase chaH 'emvo' does indicate the direction from which the army approached, but they came from there. They didn't go away from there. (Okay, perhaps they literally did, but that's not the point of the sentence.)
Based on the evidence we have I don't think we can say that a nominative object could have an ablative meaning. I think the definition just means that in some contexts (ie. when -vo' is used), the verb can be translated with "go away from".
We certainly cannot conclude from what we know that an object with -vo' must be allowed by Klingon grammar. But if you're going to translate X-vo' ghoS as go away from X with the "from" embedded in the verb instead of being part of the location, you're already treating X as the verb's object even as it bears the -vo' suffix.
-- ghunchu'wI'