The quick reply is that you can probably continue to drive it, as-is. Monitor to see if it gets worse, or other issues start. It may be worse when cold (ambient temp). Pricey fluid or additives won't help, either.
One thing you could try is shifting from P > R > D (keep in in R for a second or two before proceeding to D). However, the time spent in doing this would be about the same as going P > D and waiting the 2sec for engagement. But you could see if the engagement is smoother (I do this in my vehicles out of habit.)
As far as 'cause' of the issue... you may have low fluid pressure, or a worn boost valve. Or, your pump could be at fault (which would be contributing to the low fluid pressure.)
For the first, you'd need a transmission line pressure gauge to check. Min pressure at idle should be not less than 60psi in Drive (Reverse is higher, but you're not having a problem with that.)
The second can be done in-car - the boost valve goes in the outer pump assembly, and is visible / accessible without disturbing the valve body (it's in the orifice that the filter feed tube sits in.) However - in March of '05, GM added an Input Speed Sensor as part of this assembly, and if you have it, you'll likely have to drop the trans to remove it to, in turn, get the boost valve out (it connects in the pump body, which can't be removed in-car)
If you wanted to check the boost valve, you'd be looking for movement within the bore (fluid escapes in the gap between the valve & the bore and subsequently lowers line pressure). If present, oversized valves are available. The valve can be tricky to get out - you'll need a pair of snap-ring pliers with a long nose, as the ring is recessed within the bore, and care must be taken not to scratch / gouge the valve or the bore. I left mine alone when I put in my TransGo kit, btw, as I saw no evidence of wear.
In order for you to detect gap / movement, however, that valve / bore would have to be pretty worn; the larger valves tend to be only .001 - .002 bigger. In short, you'd have bigger problems; if it were that worn, you'd probably never engage in D.
I have a hunch that you'd find the line pressure within spec, and no evidence of a worn boost valve. If that were the case, I'd put everything back together and drive it as-is. In the absence of the issue getting worse, or other issues developing, this alone won't 'break' the trans. It would be more of an 'annoyance' issue. If it eventually does start worsening (longer and longer delayed engagement, shifts start becoming delayed / harsher), your pump will likely need to be replaced. If you go that route, and plan on keeping the vehicle, you'd be better off to do a full rebuild, not just an R&R on the pump, especially with 180K - even maintained (stock) 4L60s will start giving up the ghost about 200K or so.
Finally, one easier way to increase the line pressure is via a tune on your PCM. However, you will notice that the shifts will increase in firmness, depending on how much you boost it, and it's more stress on the pump. With your mileage, I wouldn't do this, personally.
