Fetching details from the dictionary...

remaining bony parts (of a fish after filleting)
flaw (esp. of a person); fault; defect
rice chaff
rough
crude; raw; natural; wild
「粗」 has another reading with different meanings.
魚の粗で出汁を取る。
I make stock from fish scraps.
この布は粗い織りで手触りがざらざらする。
This cloth has a coarse weave and feels rough.
市場で新鮮な粗を買ってきた。
I bought fresh fish scraps from the market.