By default GHC builds applications that open a console window when they start. If you want to build a GUI-only application, with no console window, use the flag -optl-mwindows in the link step.

For some reason, Mingw ships with the readline library, but not with the readline headers. As a result, GHC (like Hugs) does not use readline for interactive input on Windows. You can get a close simulation by using an emacs shell buffer!