If I start my own alliance how do I quit and just join someone else?

To leave an alliance that you are the leader of you need to either appoint someone else as the leader so that you step down. Or if you are the only player left in the alliance you can disband. Which will close the alliance down and leave you to join a new one.

Look in the alliance menu the manage tab for the option to close the alliance.