I was blind!
One of the first things I can choose under Configuration is, whether I would like the custom fields to be displayed by categories or events.
Yes. For someone has same question, just go to Events Booking -> Configuration, set Assign Custom Fields to category instead of to events config option to Yes and you can assign custom fields to category instead of to events
