Fetching details from the dictionary...
to go downhill (e.g. for one's business); to be badly off (economically)