Tips on the quantitative survey and evaluation method

In my research, in addition to a group discussion, I also collected the data. I carried out a quantitative survey for this. This is used for the quick collection of data that I have in the Round tables could evaluate together with the participants. In this article, I’ll give you some tips on how to do this. The following sections are taken from and quoted from my doctorate.

Advantages and pilot test

The advantage is that a high mass can be achieved quickly. The links to the questionnaire can be distributed specifically to already known participants and recommendations in their networks. In contrast to direct methods such as telephone interviews, the layout and design of the questionnaire is very important, as the researcher cannot provide explanations (Porst 2014). For example, a question can be misinterpreted. For this reason, my questionnaires were always tested with 5 pilot people, who then sent direct feedback to the research team.

Define target group

Every survey has a specific target group, which you should define. On the one hand you should limit the professional group and on the other hand justify in the thesis why you are questioning them. It also shows whether you should conduct the survey online or offline. For example, I wanted to interview disciplinary managers because I wanted to investigate leadership behavior in virtual teams. That was perfectly possible online. In another study on the digital workplace, my target group was small businesses. Most of them could hardly be reached via e-mail or the Internet, which is why we asked offline. So define and justify precisely the target group before the survey.
Tip: don’t forget yours Cleanly limit methodology.

Formulation of the questions

The exact content of the questions depends of course on your research question. However, you should precisely conceptualize the nature of the question. You derive a variable (measured variable) for each question. This can be, for example, turnover of the respondents, team size, preferred agile method, type of company or much more. The important thing is: every question creates a variable. You can then generate confirmed hypotheses from the variables. If you already have hypotheses in advance of your work, the questions should of course be adapted to the hypotheses.
If not, you can derive theses from your survey, for example, from a team size of 12 people, the interviewed managers prefer Kanban or managers from SMEs prefer Kanban and corporations prefer Scrum. The wording is based on the selected evaluation method. I’ll tell you something about this below. In a thesis, 5-10 questions are usually sufficient.
Reading tip: Prepare theses

Excursus: online vs. offline

There are two possible types of survey: online and offline. You can print out the survey and offline carry out. This has the advantage that you can reach participants whom you would otherwise not reach and that you can follow the distribution very closely. For example, a questionnaire can be distributed specifically to managers or project managers. For example, I wanted to interview exactly the same companies in a study that my co-author interviewed in 2016. We have therefore distributed the printed questionnaires specifically to them. Nobody else should take the survey. Limitation: Of course you can also protect the survey online with access codes, but this is never 100% secure.
In the Online survey you can reach a high mass very quickly, but you can hardly control how the link to the survey is distributed. Participants can also pass this on. You should therefore formulate an introductory question. In my survey of executives, for example, the initial question I asked was what number of employees the respondent currently leads. If the answer was: no guidance, the questionnaire broke off immediately. This is how you avoid wrong answers. Someone who does not lead can hardly say what managers think. In the same way, only project managers should answer a survey among project managers.

Construction and process

My online questionnaires are divided into small, thematically separated blocks and care is taken that, in addition to common answer types such as drop menus, lists, checkboxes, radio buttons, etc., there are not too many questions on one page, as too many questions can reduce the research participants’ concentration (Kuckartz et al. 2009) or an information overload for too many different topic blocks can occur for the participants. The questionnaire was therefore designed to last 15 minutes, which also turned out to be an acceptable length of time in the pilot tests.
I always schedule the surveys to last 4 weeks and have a minimum of 60 valid answers. The data is then exported from the questionnaire software and first evaluated for validity in a tool. After sorting out unsatisfied questionnaires, they are evaluated using SPSS so that the data can be visualized.
Reading tip: Book by Kuckhartz

How many people should I interview?

This is a good question, and in general, the more you ask, the more meaningful the results. My tip is that you look at the results and calculate: How significantly have the results changed since the last 5 participants surveyed. If there is no change, the results can be assumed to be stable.
To do this, add the following sentence to your work: 25 participants were interviewed. To Wilde and Hess (2006) the saturation criterion of a research method is reached if, after a certain number of participants, no significant new knowledge has been gained after an iteration. After measuring the last 5 participants, no significant new changes could be achieved in the survey.
In short: If I ask more questions, the result will hardly change, e.g. 80% prefer agile over traditional IT methods. Even if you interview 40 other participants, things should change under normal circumstances.
Overall, however, I can say as a guideline that 15-30 people were interviewed in my bachelor thesis.

quantitative survey – evaluation

After completing the simple evaluation, the data is exported from the questionnaire software. I have always examined the difference between SMEs and corporations. So I specifically wanted to find out whether, for example, special knowledge would arise for leadership in SMEs. In the SPSS analysis tool, the data is separated into the data for SMEs and non-SMEs and the responses per group are examined for a significant difference. In the check for statistical deviations, the previously specified, customary significance level of α = 5% is used (Kuckartz et al. 2013). It is derived whether a survey variable applies uniformly to all companies or whether there is a specific deviation for SMEs.

Reading tip: Book for evaluation

Four possibilities for evaluation

In addition to many other options, there are a total of four known options for evaluating large amounts of data. Of course, there are also many other methods such as difference analysis, con-joint analysis, neural networks and discriminant analysis. However, I will only describe these four procedures, as I see them most often in theses at my university.

Significance analysis

As already explained, it is described how a hypothesis deviates from the null hypothesis. Like in my example: SME and NON_KMU. This is worthwhile as soon as you interview 2 or more groups. There are differences between SMEs and NON_SMUs, for example in the number of home office days, etc. This makes sense as soon as you want to compare something or work out differences or as I specifically examine SMEs.

Regression analysis

Here you are trying to map the dependency of one independent variable on another. For example, a CEO would like to know how much money he has to invest in advertising in order for something to change in the company, such as sales figures or new customers. To do this, they create so-called scatter diagrams and see whether there is a connection between the selected variable and the others. In contrast to the next method, here the cause and effect is examined in detail. This enables predictions, which is useful for research questions when you want to make predictions.

Correlation Analysis

Here you look at the relationship between 2 variables. So whether these are related. For example, you can say whether employee satisfaction and days in the home office can be related. Does this increase or decrease with increased home office days? This makes sense when research examines an influence on something. In contrast to regression, no cause and effect is determined here, but only how similar two variables are. So here you are examining the connection in the here and now.
Reading tip: Study by me with correlation analysis

Cluster analysis

With a cluster analysis one can determine similarities in large groups and summarize them. Customer group analyzes are a great example. A marketing manager looks at which customer groups are shopping in his online shop. In a thesis, you summarize similar answers in groups. The result is then groups such as who founds startups. With the help of the cluster analysis it is possible to divide the data into groups.

Conclusion: Tips on the quantitative survey method

The method was very well suited for my research and focuses on data collection. I have used the method offline as well as online. The preparation and evaluation takes a long time, which is why every survey must be properly prepared and planned. It is also important to interview at least 50 people, since the significance tests in particular only make sense from 30 people, i.e. a total of 60 participants. My tips should give an initial orientation to the methodology. Definitely look too in my other book tips!
[student] Verwendete Quellen anzeigen