As towns go, Topsham doesn't have vast literary connections; the few I know of tend to be more by proxy, the most famous being that of the cousin and early sweetheart of Thomas Hardy, Tryphena Sparks, who in later life married a Topsham publican, and is buried here. Another such connection, coincidentally pub-related, is that Barry Stock, landlord of the Steam Packet here, is related by marriage to Colin Stanley, bibliographer of (and probably the chief authority on) the works of the philosopher and author Colin Wilson.
I first met Colin Stanley a few years back after he had achieved brief, and undeserved, minor notoriety for his self-published novel, First Novel. Personally I didn't much like it for the self-referential games (it was set in a town called Tapshed - unmistakably Topsham - and its hero was writing a novel about a fictional town called Topsham, which wasn't quite this Topsham as the street names were different ... and so on). It also had a flavour of an author using a novel for personal gripes about publishers, colleagues and so on. However, the local fuss concerned the sex scenes (in fact no more, nor more explicit, than many other modern novels) that got it banned from the Exeter WH Smith.
I first met Colin a while after this had long since blown over, found him quite unlike my expectations, and we've had a number of chats since. If you're interested in Colin Wilson's works, Stanley's Nottingham-based publishing house Pauper's Press has a large portfolio of works by and about Wilson, as well as links through to Abraxas, Paul Newman's interesting magazine of literature, philosophy and ideas, focusing on Colin Wilson.
These days, the impact of Colin Wilson's early works is largely forgotten. His The Outsider (I strongly suspect inspired from his own outsider status as a working-class intellectual) had a brief vogue, popularising existentialism, but since he has produced a huge output of variable quality: philosophy, litcrit, criminology, fiction and rather flakey New Age stuff. In later life, tongue-in-cheek self-revelations and hostile reviews from the highbrow papers - see 'Now they will realise that I am a genius' and I was a teenage nail biter in the Guardian - have fostered a current view of him as a hackwriting crank, which is highly undeserved in the light of his best works.
My particular favourites are The Strength to Dream and The Mind Parasites. The former is one of the best critiques I know on imaginative literature in the late 19th / early 20th century period that Wilson views as crucial (see the full view on Google Books). The latter is a philosophical SF/horror novel that uses, at the suggestion of August Derleth, a Lovecraftian concept of alien parasites and ancient evil to explore Wilson's central philosophies. The parasites, apparent aliens that live in the subsconscious and sap human energy, are a vehicle for an existential look at a central problem of the human condition, the inability of most of us to utilise the powers of our own minds. (Colin Stanley confirmed that it is one of the important books in Wilson's canon). See this review and the discussion of its themes in The Novels of Colin Wilson.
For more about Colin Wilson, see Colin Wilson World. The Fortean Times has an interview here that isn't a hatchet job.