Sources of Experimental Errors - Market Research

After having described the different types of experiments, we now turn to sources of potential errors in experiments.

There are several errors which may distort the accuracy of an experiment. These are briefly described below.

• History: 

History refers to the effect of extraneous variables as a result of an event that is external to an experiment occurring at the same time as the experiment.

For example, consider the design O1X O2 where O1 and O2 represent the sales affected by salesmen in an enterprise in the pre-training period and post-training period, respectively and X represents a sales training programme.

This experiment is expected to indicate the effectiveness of the sales training programme by showing higher sales in the post-training period as compared to sales in the pre-training period.

If the general business conditions have improved during the training period, when the sales could have risen even without the sales training programme.

• Maturation: 

Although maturation is similar to history, it differs from it, as the actual outcome is usually less evident.

Maturation refers to a gradual change in the experimental units arising due to the passage of time.

In our earlier example of training programme, salesmen have become more matured and more experienced due to the passage of time.

As a result, the improvement in sales performance cannot be attributed to the training programme alone.

Another example could be of consumer panels.

The members of such panels forming test units may change their purchase behaviour during the period when an experiment is on.

As the time between O1 and O2 becomes longer, the chance of maturation affects also increases.

• Pre-measurement effect: 

This error is caused on account of the changes in the dependent variable as a result of the effect of the initial measurement.

For example, consider the case of respondents who were given a pretreatment questionnaire. After their exposure to the treatment, they were given another questionnaire, an alternative form of the questionnaire completed earlier.

They may respond differently merely because they are now familiar with the questionnaire. In such a case, respondents’ familiarity with the earlier questionnaire is likely to influence their responses in the subsequent period.

• Interactive testing effect: 

This error arises on account of change in the independent variable as a result of sensitizing effect of the initial measurement. In other words, the first observation affects the reaction to the treatment.

For example, consider the case that respondents have been given a pretreatment questionnaire that asks questions about various brands of hair oil.

The pretreatment questionnaire may sensitise them to the hair oil market and distort the awareness level of new introduction, i.e. the treatment. In such a case, the measurement effect cannot be generalised to non-sensitised persons.

• Instrumentation: 

Instrumentation refers to the changes in the measuring instrument over time.

For example, consider the case when the interviewer uses a different format of a questionnaire in O2 as compared to that used in O1 .

This would case an instrumentation effect.

A similar example could be of an interviewer who in his enthusiasm and interest in the survey in O1 , explained to the respondents whenever there was any difficulty.

But the same interviewer gradually loses his interest in the survey and does not explain properly to the respondents in the post-measurement period-O2 .

Yet another example could be when sales aremeasured in terms of revenue and the company has increased the prices of its products in the intervening period.

• Selection bias: 

Selection bias refers to assigning of experimental units in such a way that the groups differ on the dependent variable even before the treatment.

Such a situation arises when test units may choose their own groups or when the researcher assigns them to groups on the basis of his judgment.

To overcome this bias, it is necessary that test units be assigned to treatment groups on a random basis.

• Statistical regression: 

Statistical regression effect occurs when test units have been selected for exposure to the treatment on the basis of an extreme pretreatment measure.

For example, a training programme may be devised only for salesmen whose performance have been very poor.

Sales increases in the post-treatment period may then be attributed to the regression effect.

This is because random occurrences such as weather, health or luck may contribute to the better performance of salesmen in the subsequent period.

Thus the effect of training programme may get distorted on account of this factor.

• Mortality: 

Mortality refers to the loss of one of more test units while the experiment is in progress.

It may be emphasised that mortality leads to the differential loss of respondents from the various groups.

This means that respondents, who left, say group A are different from those who left group B, thus making the groups incomparable.

In case the experiment pertains to only the group, mortality effect occurs when responsiveness of the respondents who have remained in the experiment differs from responsiveness of those who have ceased to be in the experiment.

Considerations for hiring Outside Marketing Research Agencies

There are several dimensions which need to be kept in view while selecting an outside agency for conducting Marketing Research.

These considerations may be enumerated as below:

1. Technical Expertise 

The marketing researcher should know who is to undertake the study and what is his proficiency in marketing research.

The client firm may find that a research agency is good at basic studies but is not competent enough to undertake complex studies.

Some research agencies are poorly staffed and as such they should be avoided.

2. Objectivity 

The question of objectivity is very important. Outside agencies should be reputable for their objective approach in research projects.

3. Confidentiality 

The client firm must ensure that the research agency maintains strict confidentiality regarding the project.

4. Economic Factors 

A client firm may invite research proposals from more than one agency.

In such a case, it would choose the most economical agency.

However, client firms should not overlook the fact that some agencies are very economical, but at the same time their quality is also poor. Quality should not be compromised.

5. Timely submission of reports 

The client firm should enquire about the reputation of the research agency especially in relation to its timely submission of reports.

Sometimes, outside agencies are quick in taking up assignments from clients but are not so prompt in carrying out the task.

6. Experience of The Supplier

The client firm should ascertain the standing of the agency. While general experience is very important, relevant and specific research experience is what should be looked for. 

7. Reputation of the agency 

It is necessary to ensure that the agency has a good reputation. 

This consideration is important for lending credibility to the research findings. 

This is of special importance particularly when the client firm intends to use the study for creating an impact. 

Since no single agency is likely to be strong on all these considerations, it is necessary that the client firm adopt a reasonable approach in this regard. 

It should ascertain which of the above criteria are crucial for its research project and then to select an agency

How Attracting Customers Made Me A Better Salesperson

Note the words that are emphasized in the paragraphs that follow.

I once had a friend from Colombia who wanted to sell coffee from his country in the Washington, D.C., area but didn’t know how to get into the market.

Finally, he succeeded on the basis of service and distribution.

He began importing coffee from Colombia and freestanding espresso machines from Italy, placed the machines in high-traffic locations, and stocked them with his Colombian coffee.

 More recently, I worked with an organization in Brazil that wanted to sell frozen açaí pulp in the United States.

Açaí is a “superfood” berry grown mainly in the Amazon region of Brazil and is very high in antioxidants.

About the time I started working with this product, early in 1996, some firms that marketed açaí products managed to have articles published in magazines, and one company arranged to have it featured on The Oprah Winfrey Show.

After that publicity, sales shot up and buyers were actually competing for the frozen pulp.

Goldman Jewelry, a company based in Israel, exports jewelry from the United States to numerous other countries.

The company’s secret is to make the product available everywhere by putting it on an online auction site.

Because Goldman’s overhead is low and its methods of promotion (eBay) and distribution (basically mail) are economical, it can offer lower prices than most of its competitors.

It nearly always helps to have the lowest prices, although this is hard for independent exporters to do

After the events of September 11, 2001, the demand for security equipment began expanding worldwide. A U.S. Department of Commerce study identified 44 ports, 47 airports, and other facilities that needed this kind of equipment just in the country of Spain.

Large and small exporters began vying for the business, and several were successful because they had products that were urgently in demand. Of course, pricing was important but not nearly so much as having products that would do the job well.

At this point I should say a word about bribery, which is an all-too-common way of influencing buyers of products and services.

It is so pervasive in many countries that the U.S. government has listed it as a barrier to trade.

I suggest avoiding this kind of activity, even if you lose the business.

First, you may be in violation of U.S. law, depending on whom you bribe and how you do it.

There is more information about antibribery legislation in the section on export regulations.

Second, if you are a beginner, you won’t know how to play the game and will probably make mistakes.

Anyone can ask you for money or favors and say that he or she can arrange for your offer to be accepted, but not everyone has the power or influence to make good on such a promise.

I suspect there are people who make their living by offering to arrange sales in return for money or other compensation and then simply say that such and such happened and they couldn’t get it done.

Third, if you pay a bribe and make a sale, you will find it very hard to avoid making similar payments on future sales to the same customer.

Once you start to give something, it’s hard to stoping it.

Of course, it is common in most businesses to pay commissions on sales.

Unlike a bribe, a commission is usually on top of (not under) the table.

Everyone knows it is being paid and accepts it as a legal and ethical cost of doing business.