Famously, on average they only take one week of the already minimal two weeks that they have available.
“A joke among some Americans who deal with Japan is that one reason Japanese white-collar workers take so few holidays is that if they were gone for more than a couple of days it might be obvious how little work they actually do”
America and the Four Japans pg 100
What could also be a factor is that there is a blurring between holiday and other kinds of days off. For example, if you don’t use your paid sick days you could theoretically take them as paid holiday, but almost no one does. This possibly reinforces the idea that time off is something to feel guilty about.