For the record, yes you need a pilot’s license to fly a hot air balloon.

And yes the “balloon police” (aka the FAA in the United States) or their equivalent governing body in other countries will stop you, and fine you.