I can't think of any reason that wanting to teach your children at home would be "selfish."
Sending children to school so they can learn test-taking skills doesn't seem like a great reason to me.
In fact, public education being what it is, I can't see a reason to send children to public school at all...but then, this is a homeschooling forum, so you might expect me to say that, lol.
But seriously, your children are still young enough that you don't have to make the high school decision yet. We can hope that by the time you do, your dh will be in agreement with you.