This is one of those changes that came about due many things taking place simultaneously.
One of those factors was food. Food storage and preparation improved. In ye olde days sailors ate and slept in the same place. Someone in each mess went to the galley and schlepped the meals back. That worked when you were bringing a lump of cheese, biscuits, and some pork in a stew. As the food got better, it made sense to have dedicated serving and eating locations. This change was also better for hygiene.
When you didn't have to eat where you sept, there was no need to fold up the hammocks to make room for a table.
The Navy's medical department raised concerns that hammocks were bad for the back.
In the early steel navy era, secondary batteries tended to be in sponsons with the guns being located in berthing areas. The berthing needed to be cleared to bring the guns into action. When guns started being placed in mounts above the main deck, the need to clear berthing disappeared.
It was possible to fit more bunks into the same space as hammocks. Bunks were typically 4 high while hammocks were usually 1 high. About thirteen feet were needed for a hammock while a bunk could be under 7 feet.
I suspect that the need for more crew in the same space was the major driver as these photos on USS Texas show.