It is also fantasy even when the author slaps a “magical realism” label on it. (Is it just me, or is most “magical realism” mostly playing a head game where the situation might actually be happening or might be happening inside the point of view characters head and therefore not “real?”)
I think that if you have such an incredible issue with writing science fiction that you have to declare that you don’t think of the science fiction piece you wrote AS science fiction that you should consider a) not writing the story in the first place or b) using a different byline so the icky science fiction cooties don’t get all over you. It seems as if you would be killing off parts of your potential fan base if you went around declaiming, “I do not think of what I write as science fiction.” I certainly don’t feel a *strong interest in an author who says something like this, and I’m generally disinclined to read their work. On the other hand, possibly there are (insecure) people who like to say they’re reading “mainstream speculative fiction,” because they are equally afraid of science fiction cooties, in which case maybe stating “I don’t write icky science fiction” is their way of alerting this demographic to their work.
*Okay, I do feel a strong interest, but that strong interest generally involves hoping the book/story gets nominated for a some kind of fantasy or sf award, hoping it wins and then Silly Stringing the writer when/if they show up for the award.