There is no central naming authority in I2P. All hostnames are local. (Naming and Address Book)
Therefore, if an url doesn't work, it isn't in your local address book.
You can add mappings yourself if you know the destination or b32 address, or make use of subscriptions. In this case, add http://i2p-projekt.i2p/hosts.txt and http://notbob.i2p/hosts.txt to your address book subscriptions, the latter contains the destination for mysu.i2p.
Ah, okay. Another question is do you have to add that or will it work once you've been on the network long enough by bouncing through other tunnels or something? Also, if I only have the host name, how do I get the B32 address? So I can just visit it directly.