As the 21st century was beginning, a South African psychiatrist named Derek Summerfield happened to be in Cambodia conducting some research on the psychological effects of unexploded land mines — at a time when chemical antidepressants were first being marketed in the country.

The local doctors didn’t know much about these drugs, so they asked Summerfield to explain them. When he finished, they explained that they didn’t need these new chemicals — because they already had antidepressants. Puzzled, Summerfield asked them to explain, expecting that they were going to tell him about some local herbal remedy. Instead, they told him about something quite different.

The doctors told Summerfield a story about a farmer they had treated. He worked in the water-logged rice fields, and one day he stepped on a land mine and his leg was blasted off. He was fitted with an artificial limb, and in time he went back to work. But it’s very painful to work when your artificial limb is underwater, and returning to the scene of his trauma must have made him highly anxious. The farmer became deeply depressed.

So the doctors and his neighbors sat with this man and talked through his life and his troubles. They realized that even with his new artificial limb, his old job — working in the paddies — was just too difficult, that he was constantly stressed and in physical pain, and that these things combined to make him want to just stop living. His interlocutors had an idea.

They suggested that he work as a dairy farmer, a job that would place less painful stress on his false leg and produce fewer disturbing memories. They believed he was perfectly capable of making the switch. So they bought him a cow. In the months and years that followed, his life changed. His depression, once profound, lifted. The Cambodian doctors told Summerfield: “You see, doctor, the cow was an analgesic, and antidepressant.”

In time, I came to believe that this little scene in Southeast Asia, which at first sounds just idiosyncratic, deeply “foreign,” in fact represents in a distilled form a shift in perspective that many of us need to make if we are going to make progress in tackling the epidemic of depression, anxiety, and despair spreading like a thick tar across our culture.

It’s not just about brain chemistry
For more than 30 years, we have collectively told one primary story about depression and anxiety. When I was a teenager and I went to my doctor and explained I felt distress was pouring out of me uncontrollably, like a foul smell, he told me a story.

The doctor said that depression is caused by the spontaneous lack of a chemical in the brain called serotonin, and I simply needed to take some drugs to get my serotonin levels up to a normal level. A few days before I wrote this piece, a young friend of one of my nephews, who was not much older than I was when I was first diagnosed, went to his doctor and asked for help with his depression. His doctor told him he had a problem with dopamine in his brain. In 20 years, all that has shifted is the name of the chemical.

I believed and preached versions of this story for more than a decade. But when I began to research the causes of depression and anxiety for my new book, Lost Connections, I was startled to find leading scientific organizations saying this approach was based on a misreading of the science. There are real biological factors that contribute to depression, but they are very far from being the whole story.

The World Health Organization, the leading medical body in the world, explained in 2011: “Mental health is produced socially: The presence or absence of mental health is above all a social indicator and therefore requires social, as well as individual, solutions.” The United Nations’ special rapporteur on the right to health, Dr. Dainius Pūras — one of the leading experts in the world on mental health — explained last April that “the dominant biomedical narrative of depression” is based on “biased and selective use of research outcomes.”

“Regrettably, recent decades have been marked with excessive medicalization of mental health and the overuse of biomedical interventions, including in the treatment of depression and suicide prevention,” he said. While there is a role for medications, he added, we need to stop using them “to address issues which are closely related to social problems.”

I was initially bemused by statements like this: They were contrary to everything I had been told. So I spent three years interviewing the leading scientists in the world on these questions, to try to understand what is really going on in places where despair in our culture is worst, from Cleveland to Sao Paulo, and where the incidence of despair is lowest, including Amish communities. I traveled 40,000 miles and drilled into the deepest causes of our collective depression.

I learned there is broad agreement among scientists that there are three kinds of causes of depression and anxiety, and all three play out, to differing degrees, in all depressed and anxious people. The causes are: biological (like your genes), psychological (how you think about yourself), and social (the wider ways in which we live together). Very few people dispute this. But when it comes to communicating with the public, and offering help, psychological solutions have been increasingly neglected, and environmental solutions have been almost totally ignored.

The hotly contested studies of chemical antidepressants
Instead, we focus on the biology. We offer, and are offered, drugs as the first, and often last, recourse. This approach is only having modest results. When I took chemical antidepressants, after a brief burst of relief, I remained depressed, and I thought there was something wrong with me.I learned in my research that many researchers have examined the data on antidepressants and come to very different conclusions about their effectiveness. But it’s hard not to conclude, looking at the evidence as a whole, that they are at best a partial solution.

Depression is often measured by something called the Hamilton Depression Rating Scale, a 17-item test administered by clinicians, where a score of zero means you show no symptoms of the disorder and a score of 52 would indicate an absolutely debilitating episode.

The studies that most strongly support chemical antidepressants found that some 37 percent of people taking them experience a significant shift in their Hamilton scores amounting to a full remission in their symptoms. When therapy and other interventions were added in addition to or in place of these drugs — in treatment-resistant cases — remission rates went higher.

Yet other scholars, looking at the exact same data set, noticed that over the long term, fewer than 10 percent of the patients in the study — who were, incidentally, receiving more support than the average depressed American would receive from their doctor — experienced complete remission that lasted as long as a year. When I read this, I noticed to my surprise that it fit very closely with my own experience: I had a big initial boost, but eventually the depression came back. I thought I was weird for sinking back into depression despite taking these drugs, but it turns out I was quite normal.

Steve Ilardi, a professor of psychology at the University of Kansas, summarizes the research on chemical antidepressants this way, via email: “Only about 50 percent of depressed individuals experience an initial positive response to antidepressants (and only about 30 percent achieve full remission). Of all of those depressed individuals who take an antidepressant, only a small subset — estimated between 5 and 20 percent — will experience complete and enduring remission.” In other words: The drugs give some relief, and therefore have real value, but for a big majority, they aren’t enough.

Irving Kirsch, a professor of psychology who now teaches at Harvard Medical School, was initially a supporter of chemical antidepressants – but then he began to analyze this data, especially the data the drug companies had tried to keep hidden from the public. His research concluded that chemical antidepressants give you a boost, above the placebo effect, of 1.8 points on average on the Hamilton scale. This is less than a third of the boost that you get, by some estimates, from improving your sleep patterns.

(Kirsch points out that the study released last week in The Lancet, to much media coverage, confirmed what we already knew and everyone already agreed on: that chemical antidepressants have more effect than a placebo. The more important questions are: by how much, for how long?)

And even people less skeptical than Kirsch point to this inconvenient fact: Although antidepressant prescriptions have increased 500 percent since the 1980s, there has been no discernible decrease in society-wide depression rates. There’s clearly something very significant missing from the picture we have been offered.

After studying all this, I felt startled, and it took me time to fully absorb it. Kirsch regards the 1.8-point gain he finds as clinically meaningless and not justifying the benefits of these drugs. I found his studies persuasive, but I disagree a little with this takeaway. There are people I know for whom this small but real benefit outweighs the side effects, and for them, my advice is to carry on taking the drugs.

But it is clear, once you explore this science, that drugs are far from being enough. We have to be able to have a nuanced and honest discussion that acknowledges an indisputable fact: that for huge numbers of people, antidepressants only provide either no relief or a small and temporary amount, and we need to radically expand the menu of options to help those people.

Our focus on biology has led us to think of depression and anxiety as malfunctions in the individual’s brain or genes — a pathology that must be removed. But the scientists who study the social and psychological causes of these problems tend to see them differently. Far from being a malfunction, they see depression as partly or even largely a function, a necessary signal that our needs are not being met.

Everyone knows that human beings have innate physical needs — for food, water, shelter, clean air. There is equally clear evidence that human beings have innate psychological needs: to belong, to have meaning and purpose in our lives, to feel we are valued, to feel we have a secure future. Our culture is getting less good at meeting those underlying needs for a large number of people — and this is one of the key drivers of the current epidemic of despair.

I interviewed in great depth scientists who have conclusively demonstrated that many factors in our lives can cause depression (not just unhappiness: full depression). Loneliness, being forced to work in a job you find meaningless, facing a future of financial insecurity — these are all circumstances where an underlying psychological need is not being met.

The strange case of the “grief exception” — and its profound implications
The difficulty that some parts of psychiatry have had in responding to these insights can be seen in a debate that has been playing out since the 1970s. In that decade, the American Psychiatric Association decided, for the first time, to standardize how depression (specifically, “major depressive disorder”) was diagnosed across the United States. By committee, they settled on a list of nine symptoms — persistent low mood, for instance, and loss of interest or pleasure — and told doctors across the country that if patients showed more than five of these symptoms for more than a couple of weeks, they should be diagnosed as mentally ill.

But as these instructions were acted on across the country, some doctors reported a slightly awkward problem. Using these guidelines, every person who has lost a loved one — every grieving person — should be classed as mentally ill. The symptoms of depression and the symptoms of grief were identical.

Embarrassed, the psychiatric authorities came up with an awkward solution. They created something called “the grief exception.” They told doctors to keep using the checklist unless somebody the patient loved had recently died, in which case it didn’t count. But this led to a debate that they didn’t know how to respond to. Doctors were supposed to tell their patients that depression was a brain disease to be identified on a checklist — but now there was, uniquely, one life situation where that explanation didn’t hold.

Why, some doctors began to ask, should grief be the only situation in which deep despair is not a sign of a mental disorder that should be treated with drugs? What if you have lost your job? Your house? Your community? Once you entertain the idea that depression might be a reasonable response to some life circumstances — as Joanne Cacciatore, an associate professor in the school of social work at Arizona State University, told me — our theories about depression require “an entire system overhaul.”

Rather than do this, the psychiatric authorities simply got rid of the grief exception.

Now grieving people can be diagnosed as mentally ill at once. Cacciatore’s research has found that about a third percent of parents who lose a child are drugged with antidepressants or sedatives in the first 48 hours after the death.

Once you understand that psychological and social context is crucial to understanding depression, it suggests we should be responding to this crisis differently from how we now do. To those doctors in Cambodia, the concept of an “antidepressant” didn’t entail changing your brain chemistry, an idea alien to their culture. It was about the community empowering the depressed person to change his life.

All over the world, I interviewed a growing group of scientists and doctors who are trying to integrate these insights into their work. For them, anything that reduces depression should be regarded as an antidepressant.

To know what to fight, we need to think harder about causes of mental malaise. I was able to identify nine causes of depression and anxiety for which there is scientific evidence. Seven are forms of disconnection: from other people, from meaningful work, from meaningful values, from the natural world, from a safe and secure childhood, from status, and from a future that makes sense to you. Two are biological: your genes, and real brain changes.

(It is too crude to describe these as a “chemical imbalance,” the typical shorthand today; Marc Lewis, a neuroscientist at the University of Toronto, told me it makes more sense to think of them as “synaptic pruning” — your brain sheds synapses you don’t use, and if you are pushed into a pained response for too long, your brain can shed synapses, making it harder to navigate away from dark thoughts.)

These scientists were asking: What would antidepressants that dealt with these causes, rather than only their symptoms, look like?

“Social prescribing”: a new kind of treatment

In a poor part of East London in the 1990s, Dr. Sam Everington was experiencing something uncomfortable. Patients were coming to him with depression and anxiety. “When we went to medical school,” he told me, “everything was biomedical, so what you described as depression was [due to] neurotransmitters.” The solution, then, was drugs. But that didn’t seem to match the reality of what he was seeing.

If Everington sat and talked with his patients and really listened, he felt that their pain made sense — they were often profoundly lonely, or financially insecure. He wasn’t against chemical antidepressants. But he felt that they were not responding to the underlying reasons his patients were depressed in the first place. So he tried a different approach — and ended up pioneering a fresh approach to fighting depression.

A patient named Lisa Cunningham came to Everington’s surgery clinic one day. She’d been basically shut away in her home, crippled with depression and anxiety, for seven years. She was told by staffers at the clinic that they would continue prescribing drugs to her if she wanted, but they were also going to prescribe a group therapy session of sorts. There was a patch of land behind the clinic, backing onto a public park, that was just scrubland. Lisa joined a group of around 20 other depressed people, two times a week for a full afternoon, to turn it into something beautiful.

On her first day there, Lisa felt physically sick with anxiety. It was awkward to converse with the others. Still, for the first time in a long time, she had something to talk about that wasn’t how depressed and anxious she was.

As the weeks and months — and eventually years — passed, Everington’s patients taught themselves gardening. They put their fingers in the soil. They figured out how to make things grow. They started to talk about their problems. Lisa was outraged to learn that one of the other people in the group was sleeping on a public bus — so she started to pressure the local authorities to house him. She succeeded. It was the first thing she had done for somebody else in a long time.

As Lisa put it to me: As the garden began to bloom, the people in it began to bloom too. Everington’s project has been widely influential in England but not rigorously analyzed by statisticians, who tend to focus on drug-centered treatment. But a study in Norway of a similar program found it was more than twice as effective as chemical antidepressants — part of a modest but growing body of research suggesting approaches like this can yield striking results.

This fits with a much wider body of evidence about depression: We know that social contact reduces depression, we know that distraction from rumination (to which depressives are highly prone) has a similar effect, and there is some evidence that exposure to the natural world, and anything that increases exposure to sunlight, also has antidepressant effects.

Everington calls this approach “social prescribing,” and he believes it works because it deals with some (but not all) of the deeper social and environmental causes of depression.

Economic stress can lead to depression

I searched out other radical experiments with different kinds of social and psychological antidepressants, often in unexpected places. (Some of these were not designed as antidepressants but ended up serving that purpose.) In the 1970s, the Canadian government embarked on an experiment in a rural town called Dauphin, in Manitoba. They told the population there: From now on, we are going to give you, in monthly installments, a guaranteed basic income. You don’t have to do anything for it — you’re getting this because you are a citizen of our country — and nothing you do can mean we will take this away from you. It added up to roughly $17,000 in today’s US dollars (if they had no income from other sources).

Many things happened as a result of this three-year experiment, but one of the most striking is a big fall in hospitalizations — 8.5 percent in three years, according to Evelyn Forget, a professor in the department of community health services at the University of Manitoba and the leading expert on this experiment. Visits for mental health reasons accounted for a significant part of that drop, Forget says, and visits to doctors for mental health reasons also decreased.

“It just removed the stress — or reduced the stress — that people dealt with in their everyday lives,” she says. There is evidence that if you have no control at work, you are significantly more likely to become depressed (and to die of a stress-related heart attack). A guaranteed income “makes you less of a hostage to the job you have, and some of the jobs that people work just in order to survive are terrible, demeaning jobs.”

The scientists I spoke with wanted to keep chemical antidepressants on the menu, but also to radically expand the options available to depressed and anxious people. Some interventions are things individuals can do by themselves. One is taking part in groups dedicated to rediscovering meaning in life (anything from a choir to a campaign group). Another is practicing a form of mindfulness called “loving-kindness meditation” (an ancient technique for overcoming envy in which you train yourself to feel joy not just for your friends but also for strangers and even for people you dislike).

But many of the most effective social antidepressants require us to come together to fight for big social changes that will reduce depression, like changing our workplaces to reduce the amount of control and humiliation that happens there.

As a 39-year-old gay man, I have seen how people can band together to fight for seemingly impossible goals — and win, radically reducing the amount of unhappiness gay people face. I have also seen how, in one sense, the struggle is the solution: The act of banding together, identifying that you are being mistreated, and fighting for something better restores dignity to people who felt they had been defeated.

Is there a type of depression utterly unconnected to life circumstances?
As I absorbed all this evidence over three years, a persistent question kept coming to me. Yes, there are these deep causes of depression, but what about people who have nothing to be unhappy about, yet still feel this deep despair descend on them?

There is a debate among scientists about whether there is something called “endogenous depression” — a form of despair that is triggered purely by biology. The most detailed research into this, by George Brown of the Institute of psychiatry at the University of London and his colleague Tirril Harris, in the 1970s, found that people diagnosed with this problem in fact had just as many life challenges as people whose depression was supposed to be a response to life events. (They had spent years studying how long-term stress can radically increase depression.)

This could mean that endogenous depression does not exist — or it could mean that scientists were not good at spotting the difference back then. The scientists I spoke to agreed on one thing: If the condition does exist, it affects a tiny minority of depressed and anxious people.

But I only really felt I made a breakthrough in my own thinking — in understanding the mystery of why some people seem to become depressed “for no good reason” — when, by coincidence, I started reading some feminist texts from the 1960s.

At that time, it was common for women to go to their doctors and say something like: “Doctor, there must be something wrong with my nerves. I have everything a woman could want. I have a husband who doesn’t beat me, two kids, a house, a car, a washing machine — but I still feel terrible.” Doctors would agree that they had a problem and would prescribe them drugs like Valium. (The locus of the problem only migrated from the “nerves” to the brain in the 1990s.)

Now if we could go back in time and talk to those women, we would say, “Yes, you have everything you could possibly want by the standards of the culture.” But the standards of the culture are simply wrong: You need much more than this.

In the same way, today, when people tell me they must be biologically broken because they have “everything they could want” yet they are still depressed, I say: Tell me what you have. They talk about having money, or status, or expensive consumer goods. But these are not what people need to have meaningful lives.

If I start to ask about the social and environmental factors of depression and anxiety I’ve mentioned, I have yet to find a depressed person for whom at least some are not playing out. Perhaps some of us are simply biologically broke, but the idea that a purely biological story describes the vast majority of depressed and anxious people is by now, it is fair to say, discredited.

The lesson the psychiatrist took back from Cambodia
After he had completed his work in Cambodia, and after he had heard the story about the farmer who was given a cow as an antidepressant, Summerfield returned to London, where he worked as a psychiatrist, and he realized something he had never quite seen so clearly before. He thought about when he had most helped his depressed and anxious patients. Most often, it occurred to him, it was when he helped them to get secure housing, or to fix their immigration status, or to find a job. “When I make a difference, it’s when I’m addressing their social situation, not what’s between their ears,” he told me.

Yet we have, as a society, built our responses to depression and anxiety almost entirely around changing brains, rather than changing lives. Every year we have done this, our depression and anxiety crisis has got worse. When, I began to wonder, will we learn the lesson that those Cambodian doctors understood intuitively, and that the World Health Organization has been trying to explain to us: Our pain makes sense.