Fetching details from the dictionary...

don't get me wrong, but ...; I'm sorry
不悪、今回のご依頼にはお応えしかねます。
I'm sorry, but I am unable to fulfill this request.
不悪、率直に申し上げますがその案には賛成できない。
Don't take this the wrong way, but I cannot agree with that proposal.
不悪、本日は先約がございますのでご遠慮いたします。
I'm sorry, but I have a prior engagement today, so I must decline.