Kaoru doesn't find girls attractive, very unlike his twin Hikaru.

Hikaru gets crushes on cute class reps or cute girls in general. Sometimes, Kaoru thinks, his twin finds the girls that go to the Host club cute too.

In a superficial way. Hikaru has stated on many occasions that the girls are annoying, but cute. If Kaoru had the same stance as 

Of course, saying Kaoru never found any girl to be attractive was a lie.

He finds Haruhi quite adorable. Of course, when she's wearing masculine clothing. Haruhi in girls clothing is...

But, it's not like Kaoru's gay or anything. He just hasn't found the right girl, that's all. Just because he's not like a flirt like Hikaru, doesn't label him gay. 

(And just because he thinks Kyoya is hot, also does not make him gay.)