Do plain regular t-shirts go with boat shoes?

Or do you have to wear a polo or button down (short sleeve)? Ive never worn boat shoes but i like the style, they are considered casual shoes right? A white t-shirt and khaki cargo shorts will work fine with them or not?