There's just so much awesome stuff that you can do with the PSP. I definitely plan on buying a 2GB Memory Stick so that I can... Do something with it. I'll be buying an MP3 player sooner or later (later is likely), so I don't really need to transfer MP3s onto it. What I may do is transfer a few movies onto the memory stick. Movies not released on UMD. Either that or perhaps some Adult Swim shows. I could definitely fit the first volume of Aqua Teen Hunger Force onto a memory stick.
I do have some tiny gripes, though. The screen is so big, it's a wonder why Sony didn't add on a flippy-down thing, one like the DS and SP have. It'd really protect the screen well. Also, why only one analog stick? This is a small complaint, but there's definitely room for it.
