I have been "adjusting" my rear brakes lately when I get home by letting it roll back a bit then slamming the stop pedal.

Today I noticed with the truck off that when I stop when rolling back in neutral there is a rubbing/grinding/squeal noise coming from the rear drivers wheel (might be the passenger wheel too but I can't hear on that side)

Is that just the drums making noise?

They don't do it when I slow down easily.

Just when I jerk them by stopping abruptly.

Brakes are good, truck stops like a champ.

Should I quit adjusting them?

 

I also want to note that the engine is off when I hear it.

 

Just pull a drum and inspect it. It could have a rock in it.

 

Everything looks good - is it just the brakes grabbing with the truck off? Lack of brake pressure?

 

Paging infidel

 

Good advice. Looking is always a good place to start.
Could also be a broken spring.

 

I looked and don't see anything...

 

And why is it so pronounced when the truck is off?

 

Well, you ain't got other noises to cover it up. There is a small amount of movement in brakes as they apply. Also where shoes rub backing plate is noise prone. Sometimes sanding the linings and cleaning dust from drum helps.
Don't force brake adjustment.