I have a bad habit of reaching for a div for everything too, but you can improve the structure of your HTML bit by bit, you don't need to rewrite the whole thing all at once. Think of it as using the right element for the job. <main>, <section>, <header>, <nav>, and <footer> are good places to start.
There are accessibility and SEO benefits, but imo just having more readable HTML is a good enough reason on its own. Of course you don't have to do anything differently if what you're currently doing is working for you. :) One of the best things about HTML and browsers are how flexible they are.