Hi
It is not a bug. It is how it is designed to work. When you choose to pay with Paypal but don't make payment, you are not registered in the system. So it won't be displayed in the front-end and in registration history.
It is also not displayed in back-end by default. It is only showed if you set "Show Pending Registrants" config option to Yes (I think you set to Yes and that's the reason it is showed).
Hope the explanation is clear to you.
Regards,
Tuan