Heroic Skeptical Odysseys Into Parapsychology Part II

Scientific Method Challenges Extraordinary Claims: Part II

(Image is of young skeptics in action)

Summary of Part I

The purpose of this two-part article is to present an eight-step method that a self-organized community of scientific skeptics developed specifically to investigate the claims of the existence of paranormal phenomenon. I began Part I with a survey of what the Yankee population believes about the paranormal, including psychic healing, ESP, the existence of ghosts and extraterrestrials’ visitations to earth. Using the work of Jonathan Smith I categorized eight types of paranormal experience on a spectrum, based on how modest or extraordinary their claims are. The bulk of the article was to name the first three of the eight steps skeptics use in evaluate paranormal claims. This include the quality of sources, how sound the logic is as well as the quality of observation.

In part two of my article, I complete the rest of the eight steps. These include quantitative reasoning (assessing probability); perceptual trickery; memory errors; the placebo effect and the possibility of hallucinations that can lead to false conclusions about the paranormal.  Furthermore, I identify four reasons why believing in the paranormal is not harmless, either at an individual or societal level. I close my article with the optimal conditions for a good scientific process of investigation as well as how best to evaluate the most superior among competing theories. The image in the front of this article are young skeptics in action.

Estimating Probability and Gauging of Chance

Are you more likely to die on a motorcycle or on a bicycle? Are you more likely to die on a bus or a train? Which is more likely, drowning in a swimming pool or in a bathtub?  A psychic will claim it is big news that among 75 people in a room, she predicted there are two people there who have the same birthday. Did you know that the chances of this happening are 99%?

People around the world are very bad at gauging chance occurrences. This is because quantitative rationality only goes back to the 17th century. Learning this skill takes special training in at least two courses: research methods and statistics. We also misjudge probabilities because of a lack of experience with the unusual which only becomes possible through familiarity working with large numbers.

Heuristic mistakes

People are also terrible at estimating probabilities because we are prejudiced for  numbers which stand out and are easy to recall. This is called the availability bias. Conversely, people underestimate the probability of rare negative events such as the likelihood they will get injured in a car accident. Furthermore, we have slight biases towards optimism rather than seeing things as they really are. This is why smokers think they are less likely to die of cancer than other smokers. A knowledgeable but unscrupulous psychic or astrologer who knows these human tendencies can comfortably predict that you will have good fortune to persuade you that astrology works.


Coincidences are events that unexpectantly occur together in a meaningful way without any apparent causal link. Therefore, they could be interpreted as a prophesy or an omen. The average person is very bad at generating random numbers. This is because random numbers are just as likely to appear in clumps or streaks as well as simple alternations. Alternations like HTHTHT to the end of a page are too regular to be random. They need to include streaks like HHHHH or TTTTT. People underestimate the frequency of the size and the frequency of the clumps. Winning at gambling 3 times in a row is just as usual at alternating between winning and losing. For our purposes it is important to know that Blackmore and Troscianko have found that people who believe in the paranormal ability are especially prone to make mistakes in probability judgments. This is called psychic bias.

Perceptual Trickery

Barnum Effect

Perception (how our senses are organized) is selective and distorted. We select (mostly unconsciously) what serves our interest and needs. Our perception is something like a spotlight that targets and intensifies some stimuli and ignores others. Our emotions and motivations guide this spotlight. Tricksters can, and do, take advantage of this. The Barnum effect is displayed in the following experiment by Bertram Forer’s personality tests. He gave participants generic descriptions based on their horoscopes and found that subjects who were more likely to believe the results were true if: a) the subject thinks it only applies to them; b) the subject has confidence in the authority; and c) the horoscope reveals mainly positive traits.

Cold Readings

Stage magic is an old practice in which skilled sleight-of-hand is involved in deceiving an audience into making errors in perception. The range goes from simple card tricks to escaping locked jails to the feats of Houdini. Stage magicians are so good they have convinced PhD physicists under laboratory conditions that paranormal effects are real. For instance, that they can bend spoons and read pictures sealed in envelopes. Cold readings are a judgment about a stranger based on a combination of reading body language, knowledge of rhetorical techniques, behind the scenes investigation into a person combined with knowledge of how to create atmospheric effects.

Jonathan Smith names five techniques in cold reading.

  • Maximize the Barnum effect and confirmation bias

You pursue elaborations of a prediction that seem to evoke a positive response in the audience. For example, “you have more creative talent than you give yourself credit for” or “your friends respect and love you more than you might expect”. After that, engage in “shot gunning” – you ask your subject so many questions and claims that some are bound to be right. Participants are more likely to seize on what fits and ignore that what doesn’t fit (confirmation bias).

  • Develop sneaky strategies for tricking the subjects into telling you things about themselves when they are not focused

One method can be informally chatting with the audience before a performance is about to begin. These can easily be worked into part of the reading. Another is to encourage cooperation. Insist that doing a reading is a collaborative venture. This is a way to narrow down the scope of what the stage magician is after. For example, games such as Twenty Questions where the question asked is “is it an animal, vegetable or mineral?” allows the narrowing of the focus through the process of elimination.

  • Ways of drawing inferences from information other than what the subject tells you

For example, reading subtle cues and body language, noting the clothes, demeanor, posture and gestures of the subject. Also, what really captures attention is basing predictions on a probable but unexpected statistic. For example, people don’t realize that in most homes they would likely find old, unsorted photographs; some toy or book that dates from childhood; some jewelry from a deceased relative; a deck of cards with one or more cards missing and an instruction book on a hobby one no longer pursues. When the psychic mentions one of these the naïve target begins to wonder whether there really is something to parapsychology.

  • Ways of making less than perfect readings seem accurate

An easy way to turn a miss into a hit is to say your claim refers to something that will happen in the future. Another technique is to blame the subject for being too skeptical. “You are thinking so much it is blocking the energy” or “there is too much negative energy in the room”.

  • Creating atmospheric effects

The hypnotist, Mesmer, was very good at creating atmosphere such as candlelight, incense, eerie or dreamy music and photos of mystics and strange animals, especially cats.

Memory Errors

This is especially relevant when it comes to paranormal tales of reincarnation and alien abductions. Many people believe that everything we experience is recorded in memory, like a personal security system camera that is always with us. In fact, our memory capacity is limited and new memories can replace corrupt old ones. Memory is reconstructed more as historical fiction or docudrama. Also, memories such as alien abductions could hardly be repressed as advocates suggest. Traumatic experiences are much more likely to be remembered.

A false memory is an inaccurate recollection based on selective forgetting – the mixing of memories or memory fragments, along with the mushing together of dreams and fantasies. In addition, we don’t have mechanisms which can isolate our real past from information gotten from TV and movies. Neither can these memories be isolated from the influence of authority figures like therapists who are committed to theories of repressed memories.

Placebo Effect: When Wishing Makes it So

What it is and when it works

What is a placebo? A placebo is when a pharmacologically or physiologically inactive substance can have therapeutic, physiological or psychological effect if it is administered to a patient who has the expectation that it is effective. The placebo is stronger for problems with a strong psychological component, such as pain or depression with up to 60% efficiency. As you might expect, placebos have less effect on problems that are mostly physical, such as cancer. The following are some tips for how accentuate the placebo.

How to pump up your placebo

  • Motivate your subject to want to get better. Give then exciting testimonials of others.
  • Find a nurse to administer the treatment.
  • Give a complicated explanation that sounds plausible and uses scientific sounding terms.
  • Introduce a complicated and sophisticated procedure. Put water in a chemistry flask surrounded by tubes that run through an electronic device with lots of knobs, dials and lights.
  • Belief that colored pills work better than white pills; large rather than small pills; capsules are better than pills. They work better when they are administered frequently rather than infrequently and when they are expensive rather than cheaper.
  • Alter the placebo so that it has a slight negative side effect. Spike the water so that it stings. Or give it a slight unpleasant medical odor. The sting will be interpreted as a sign it is working.


What they are and how sleep effects them

Hallucinations are generally defined as false involuntary perceptions that occur while awake when a sensory experience occurs in the absence of corresponding external stimulation. Before falling asleep some people experience hypnogogic hallucinations which are typically static images. On the other hand, hypnopompic hallucinations can emerge in the twilight state of sleep but before waking. Experiences of ghosts, alien inductions and angels reported throughout history and around the world are examples of what can happen when hypnopompic or hypnogogic hallucinations or sleep paralysis in physiologically is in operation.

Hallucinogenic proneness

Smith cautions us that while some people are more likely to experience hallucinations than others, it is a mistake to think of some sort of hallucination trait, once manifest, can affect someone for the rest of their lives. Instead, Aleman and Larøi prefer to use the term “hallucination proneness” which may be expressed in childhood is controllable and emerges only when triggered.

Five types of hallucination;

  • Deprivation which can include food and fasting; oxygen deprivation (too much or too little carbon dioxide); sleep deprivation and fatigue
  • Reduced sensory input

Sensory loss (blindness or loss of hearing) or social isolation

  • Sensory overload

This includes increased external stimulation such as a gambling casino, an amusement park or walking in Manhattan’s Times Square.

  • Sensory orchestration – to create a depthful meaning. A sacred example of this is in magical rituals with singing and dancing. A Catholic religious ritual which combines stained glass windows (sight); organ music (sound); incense (smell) oak pews (touch) and holy communion (taste). A secular version of this would be creating a patriotic spirit with songs, pledging of allegiance, taste (apple pie) and flags.
  • Stressful and strenuous situations

Such as trauma and bereavement.

  • Consumption of chemical substances

These include LSD, cannabis, mescaline, PCP, amphetamines and cocaine.

Please see the table at the end of this article for a summary of the typical mistakes parapsychologists make in each of the eight steps of critical thinking.

The Bottom Line About Parapsychology

Why the public likes parapsychologists

In spite of all these criticisms, why does the public continue to believe in the paranormal? Some of this has to do with the way scientists are perceived. The public’s picture of a scientist is they are overly serious and somewhat angry that people  remain interested in parapsychology. They seem narrow and rigid, as Smith says, “sexless geeks wearing white lab coats, isolated from the real world in windowless laboratories”. Scientists speak in a specialized language of mathematics which takes time to learn. On the contrary, books about parapsychology are filled with mystery and can be understood by an educated layperson.

When it comes to physical health, most medical doctors are locked into managed care hospitals with little time for their patients. But if the public wishes to see a homeopath, they will be attended to by a professional who is not beaten down by seeing patient after patient. Homeopaths are more emotionally available and will see the patient for a full hour. Attention, as we know, is very important to people. Parapsychologists seem to be more open and flexible, although as we shall see they are actually less open.

Parapsychologists as researchers

Jonathan Smith concludes that Psi researchers are frustratingly persistent in ignoring and discounting reasonable requests to fix problems. Replications have been inconsistent and are generally cancelled out by other studies that find no effect. Psychologist James Alcock says that parapsychologists have never been able to produce a successful experiment that neutral scientists with appropriate skill, knowledge and equipment have that can replicate the original experience.

Meanwhile, parapsychologists complain that mainstream science is inflexible, prejudiced and unconsciously afraid. They imagine that scientists are not open to the mysterious. But this is not true. Smith points out that in the case of dark energy it was once rejected and it took five years of astronomical observations to change their minds.  But they did change. Science is open to new findings if it plays by the rules of scientific method, as we shall see. In fact, some of the findings of science such as Einstein relativity theory, black holes and dark matter are far weirder and more exotic than anything parapsychologists have come up with.

Who is more rigid in changing their minds, believers or skeptics?

Psychics often claim that skeptics are rigid in their thinking and not open to new possibilities. But research by Susan Blackmore has not born this out. She says research shows that when believers are compared to skeptics in terms of changing their minds, skeptics are more likely to change. Believers are rigid and fail to reconcile the negative evidence.

Paranoia among parapsychologists

The distance between parapsychologists and mainstream science can be seen when parapsychologists often argue that some individual, agency or force is actively suppressing evidence. For example, the state doesn’t want you to know all the proven powers of the human brain because such knowledge could lead to challenging the state. Therefore, people believe in ESP. Another example is about flying saucers. Information that flying saucers have visited earth would case mass panic and social chaos. Therefore, the information is being suppressed. The movie Close Encounters of the Third Kind is an example. Others will say there are some everyday herbs that can cure many illnesses and eliminate the need for physicians and specialist treatments. The medical profession hides information that could threaten the livelihood of doctors.

It is frequently noted that one of the strongest findings in scientific research on the paranormal is a negative correlation between study quality and obtained support for the paranormal. In parapsychologypoorly designed and inexpensive studies are more likely to yield positive results. Good studies are much more likely to yield negative results. This is the opposite of what is found in all other areas of research. There, the better the studies the more likely we are to find a positive result.

Dangers of Believing in Parapsychology


Some may say to scientists “what is the harm of believing in ESP or telepathy? It isn’t harming anyone?”. Well the answer is it can be quite harmful. For one thing the flying saucer cultists of Marshall Applewhite paid with their lives by collectively committing suicide. Nine hundred followers of Jim Jones lost their lives when they followed his instructions to kill themselves. Followers of fundamentalist David Koresh died in a shoot-out in Waco Texas

Religion and politics

Those Catholics and Protestants who believed in the realities of witches in the Middle Ages murdered at least 100,000 witches. In the case of faith healing, people throw crutches away and the pain returns after the adrenaline rush subsides. Seriously ill patients have died after paying for healing instead of medicine. Parents have lost children they have deprived of life-saving medicine by those who followed Christian Science claiming all disease is in the mind. During the Nazi murders of Jews in the 1930s, the Nazi top advisers consulted astrology.

Costs of alternative medicine

Smith points out that up to 75% of the population believes alternative medicine is as effective as mainstream medicine. The risks involve the cost of ineffective intervention. In the case of acupuncture research has shown that its successes mostly have to do with the placebo effect. If acupuncture is used alongside of Western medicine its limits are difficult to expose. But if a person uses acupuncture instead of western medicine the costs are very high since most health insurance doesn’t cover it and the results are questionable.

Costs of subjective relativism: believing every individual creates their own reality

Subjective relativism is the claim that each of us creates our own reality. Reality depends on what you believe. For upper middle class people who become interested in the paranormal, specifically channeling, the cost of imagining you create your own reality is not high. Since upper middle class people have a high degree of control over their lives it is not so far-fetched to believe this. But suppose the person is from the working class background and cannot find work because of the economy. They would be completely justified in getting depressed because they are unsuccessful in creating their own reality. It would lead to political passivity and hopelessness because for the working class, socio-economic reality is much harder to change.

Problems with subjective relativism are as follows:

  • If an individual can will something to be true, someone else can will it to be false.
  • There is a logical contradiction: if in the real world there are no absolutes; if subjective relativism is part of the real world then it is also not absolutely true.
  • They attribute to themselves god-like powers – individuals make anything exist by thinking or wishing it. For most people in the world this may seem grandiose, to say the least.
  • They are eclectic – they don’t take a stand on the validity of conflicting claims. This leaves the contradictions to lay in suspended animation.
  • They appeal to ignorance. They say just because it can’t be measured by science doesn’t mean it’s not true. They do not provide any way other than science that can be tested beyond personal experience.

What is a Good Theory?

Public, replicable, reliable and valid

All observation involves collecting data in a way that is both public and replicable.

Typically parapsychologists have great difficulty producing these conditions because they were not public. Secondly, they were not replicable because there was no way the researcher could do the same thing in their head. In addition the measures should be both reliable and valid. Reliability means consistency of results over time. That means yielding the same result again and again. Validity measures what it claims to measure and not something else.

Free of fraud, error, deception and sloppiness

The optimal research situation needs to have an expert, independent and impartial supervision and replication to eliminate fraud, detect error, deception and sloppiness.

Fraud means the investigator changes data, reports only positive results, fails to report compromising design features, or claims to have done something they haven’t done.

Error is when the investigator misuses experimental tools, methods or statistics unintentionally. Deception is where research participants, assistants or colleagues trick the investigator. Lastly, there is sloppiness. This is where the investigator does not account for stimulus leakage, untrained or careless assistance or failing to account for eight steps in critical thinking that skeptics use. After publication, the mechanisms of science are unsuited for identifying potential sources of sloppiness, error and fraud. For example, laboratories may have closed making direct inspection impossible. Out of the thousands  of experiments done on paranormal  claims few provide enough evidence to check for sloppiness, error and fraud.

Control groups, double-blind procedures and stimulus leakage

The best way to rule out alternative explanations is to establishing a control group, have in place double blind procedures and controls for stimulus linkage. What do all these mean? Control groups answer the question – compared to what? A good way to rule out random fluctuations as an explanation is to include a control group. Perhaps students generally pass their first driving test after hot herbal tea for a week. The claim might be that the herbal tea has calming ingredients.  We need a control group of students who take their first driving test without drinking herbal tea and seeing the difference.

Stimulus leakage can come about when the enthusiasm and beliefs of an experimenter rubs off on participants. Perhaps they have unconsciously picked up on this excitement and become more motivated. In addition, a psychic might detect the facial expressions of their students. This is hard to test and may require a magician or any expert trained in deception, distraction and subliminal control. To protect against this leakage double-blind control might be introduced where neither the experimenter nor the participants know which treatment anyone has received.

Criteria for an Adequate Theory

In their book How to Think About Weird Things, the authors identify five criteria for an optimally adequate theory.


This means a theory is formulated in a way that it states the conditions under which it can be proven wrong. In the field of the paranormal Jungians are often guilty of failing the falsifiability test. Their theory of the existence of the collective unconscious is not stated in such a way that you could find out if it were real one way or another. When alternative medicine people discuss “energy blockages” they don’t present their claims so that the existence of this energy can be measured and challenged. Quantum theory of ESP cannot be tested. How can we test the existence of connections between thoughts connected to a subatomic level? If everyone is connected shouldn’t everyone have access to everyone’s thoughts all the time? Or are the connection between thoughts random, like the flickering of electrons? In the case of astrology, predictions can be falsified but the problem is that the astrologers simply don’t take nature’s “no” for an answer. The complexity of the system allows for a slew of never-ending secondary rationalizations.


Suppose two theories are testable. How do you decide which theory is better? Fruitfulness not only explains existing phenomenon but it explains phenomenon in another field. A very good example of this is Darwin’s theory of biological evolution. Not only has Darwinian theory explained the life of animals on earth but it also explains certain aspects of social evolution as well. Specifically, the sexual behavior of men and women in mating strategies.

Comprehensiveness (scope)

How much of the world within a discipline does the theory explain? Good theories have wide scope. Einstein’s theory of gravity is more productive than Newton’s. It predicted the unexpected. It includes more than what Newton provided without violating Newton’s theory which was still true on a smaller scale. In addition,

another question to ask is if it has evolved over time. Parapsychology is very weak in this area. After a century of research, it is still asking the same questions. There is no development of deeper questions after the basic questions have been settled.

Simplicity (Occam’s razor)

The best explanations are ones the requires the fewest assumptions. A weak theory implies additional untested questions that must be answered. Also, the link between the parts of the theory are not explained simply. Assume no more than is required to explain the theory. The simpler the better and the easiest to test. If possible, it is better not to use a blow furnace to light a cigarette!

Conservativism (preservation and respect for the past)

Some parapsychologists announce that their findings will revolutionize the sciences, mostly physics and psychology. They talk about this as if it were a good thing. The fact that all the hard work of centuries that have been built up would be overturned is a signal that the parapsychologists’ theory is not likely to be true. Because parapsychologists often see themselves as outsiders undermines their sense that they are part of a community. So do they celebrate some kind of victory? Sometimes parapsychologists are not very well informed about the disciplines of physics or psychology and don’t realize what has already been achieved as well as what has proven wrong. A theory is  better if it does not create more problems than it resolves.

On the whole, maximum adequacy explains a wide ranging set of phenomenon in depth within a discipline (scope) and across disciplines (fruitfulness) which doesn’t conflict with what we already know (conservativism). It does so with the simplest explanation that can be tested to yield results beyond a reasonable doubt.

The Challenge of Pandora’s Box

One problem that scientists face is how to deal with the relationships between the spectrum of the paranormal I have laid out. In many of my critical thinking classes over the years I asked students to fill out a paranormal profile. What I found is that some of them believed in ghosts, but they thought that extra-terrestrial abduction was too far-fetched. Others felt the opposite. Some believed that ESP was reasonable but poltergeists weren’t. What they didn’t realize is that their ontologically and epistemology were inconsistent. If we return to the levels of nature we discussed at the beginning of this article we found:

In science there are three levels in nature:

  • the world of physics material objects that have volume, occupy space and can transmit physical energy;
  • the world of biology in which living creatures require food, reproduce, transmit disease, get sick and die; and
  • the world of psychology in which living beings have choice, make decisions and have a conscious mind.

What parapsychologists do is cross these boundaries like:

  • mixing physics and psychology, claiming that rocks can have thoughts and that thoughts can move rocks;
  • mixing physics with biology, claiming you might get sick by standing next to a broken rock; and
  • mixing biology with psychology. This claims you can make someone sick by thinking about them.

Since we didn’t discuss alternative energy treatments or supernatural religion we have six areas on the paranormal spectrum. They all cross ontological boundaries. What Jonathan Smith says is that if we open the paranormal box just enough to let one out we must accept the right of others to come out. If you critique one thoroughly it must apply to others and all the rest. If you believe in ghosts you must believe in astrology and reincarnation. This puts the paranormal proponent in a bind. If they accept all the paranormal beliefs they will be caught in contradictions which my students would have to face, especially the contradictions between parapsychology and supernatural religion. If they accept opening Pandora’s box they must live with beliefs that might not make intuitive sense. If you accept weak support in bad sources, bad logic and sloppy scientific observation – and you wish to be intellectually honest you must accept other claims based on equivalent support. Accepting a claim that would have catastrophic implications for physics such implications should not prevent you from accepting other equally catastrophic claims.

Summary of Paranormal Common Mistakes

Paranormal Common Mistakes Example
Ontological Fusion

Mixing physics and psychology

Claiming that rocks can have thoughts and that thoughts can move rocks
Mixing physics with biology Claiming you might get sick by standing next to a broken rock
Mixing biology with psychology Claiming you can make someone sick by thinking about them
Questionable sources Claiming “ancient wisdom”
  Relying on testimonials, anecdotal evidence
  Fallacy of popular wisdom
  Low standards of mass media and internet
How sound is the logic?
Confusing necessary with sufficient conditions If the stars and the planets are aligned properly you will recover from your cold quickly

You recovered from your cold quickly therefore the stars have been aligned properly

Fallacies in chain arguments All atoms possess some gravity

Our brains are made of atoms

Thoughts are generated in the brain

Thoughts travel by means of gravity (not true)

Thoughts that travel to other dimensions return to our dimension instantaneously (not true)

Use of weasel words Fusing psychological healing with physical healing
Equivocation Fusing faith in religion with confidence in scientific laws in science
Appeals to ignorance No one has disproved UFO sightings; therefore they must have happened. This ignores naturalistic explanations.
Confusing correlation with cause Growth of interest in the paranormal in large cities and a rise in the number of cults

Believing the paranormal causes cults to rise instead of understanding that growth of cities produce both the paranormal and cults.

How trustworthy are the senses? Channelers in a darkened room

UFOs in a quiet sky

Haunted house investigators sitting in a basement at night

Ignoring the autokinetic effect

Ignoring the effect of pupil dilation

Innumeracy in gauging chance

Inability to generate random numbers

Overestimation of positive possibilities
  Underestimating negative rare events
Perceptual Trickery


Barnum effect Vague statement where the individual thinks the prediction is just for them

If they think the person is an authority

If it is stated positively

Susceptibility to cold reading techniques About ESP, clairvoyance or astrology
Memory errors


Relating to memory as if it were a tape recorder and not a reconstructive combination of stories, movies, dreams
False memory syndrome Applies to alien abductions and reincarnation claims
Underestimating the placebo effect

When wishing makes it so


Talking to the dead

Near death experience



Hypnogogic –before falling asleep Ghosts, alien abductions, angels
Hypnopompic —before waking Ghosts, alien abductions, angels
Sleep paralysis Ghosts, alien abductions, angels



About Bruce Lerro

Bruce Lerro has taught for 25 years as an adjunct college professor of psychology at Golden Gate University, Dominican University and Diablo Valley College. He has applied a Vygotskian socio-historical perspective to his five books: "From Earth-Spirits to Sky-Gods: the Socio-ecological Origins of Monotheism, Individualism and Hyper-Abstract Reasoning", "Power in Eden: The Emergence of Gender Hierarchies in the Ancient World" (co-authored with Christopher Chase-Dunn), "Social Change: Globalization from the Stone Age to the Present", "Lucifer's Labyrinth: Individualism, Hyper-Abstract Thinking and the Process of Becoming Civilized", and "The Magickal Enchantment of Materialism: Why Marxists Need Neopaganism". He is also a representational artist specializing in pen-and-ink drawings. Bruce is a libertarian communist and lives in Olympia, WA.

View all posts by Bruce Lerro →

Leave a Reply

Your email address will not be published. Required fields are marked *