In this example, I've created a close button with a div and a button without any extra code. Try them out with a keyboard.
If you don't think about accessibility, the div-based option might seem just fine. But unless you've created custom code for the element, keyboard users can't access that div element... which is quite annoying if this element is used to close, for example, a huge automatically opening modal form.
Just by using a button instead of a div keyboard users can access the element. No extra code is needed.