Hello Jodi,
From the photos and your description, It looks like more of a fungi problem than just wood-boring insects. In other words, I think it may be a dry-rot situation that will attract larger wood-eating insects.
I would consider it seriously enough to consider an intervention. Without a physical examination, I can not propose a specific treatment. However, it will need to be treated with borates and then consolidated.
Take a look at this article on treating architectural rotted wood for some insight:
Epoxies for Wood Repairs in Historic Buildings Craig
------------------------------
Craig Deller
Fellow
------------------------------