Request further information from MMSI:
Contact Us.
Introduction to trend analysis
The Dynamic Reporting Trend Analysis function enables users to generate time series graphs in real-time,
benchmark the performance of SBUs, and tailor data for specific reporting needs.
From this platform users can intuitively progress from macro level dash board reading of Key Performance Indicators
to micro level case specific respondent information and question responses, including open comments.
Where interviews have been recorded users can navigate to soundfile recordings, turning a quantitative piece of
transactional research into qualitiative understanding of respondent issues.
One table is produced for each of the series selected generated. The reporting unit appears as a heading on the relevant table.
Period - presents in tabular form a list of reporting periods within the parameters specified in the
trend analysis screen.
Samp - short for sample, this field shows the number of surveys from which scores have been calculated.
Minimum and Maximum values are applied to calculations. For example, on a 1 to 5 satisfaction scale scores outside of
this range are excluded.
Mean - this is a calculation of the average score for all surveys for this reporting unit that fall between the minimum
and maximum allowed range of response values.
Value - this is a percentage score that conforms to specific project / client requirements. For example, a Top 2
score can be derived from the proportion of respondents who answered 4 or 5 out of the total number of respondents
that answered with a valid score of between 1 and 5 inclusive.
Detail - generates a histogram, profiling the distribution of responses for that reporting unit.
From here users, if permitted, can access respondent information. Click on the blue 'Go' button to view case specific
information.
This graph shows the distribution of response codes for the selected reporting unit. The legend below the graph
shows the exact number of respondents that answered each of the response codes that fall within the valid range.
The title of the chart includes the selected attribute, author, and date of production. Be aware that there are
fundamental cultural differences in the distribution of response codes, which must be considered when interpreting
and drawing performance related conclusions for different geographical areas.
This table contains respondent profiles in a format agreed with the client during the set-up phase.
25 survey responses are displayed on each page. Use 'Next' and 'Previous' to navigate through pages.
In addition to respondent profiles, this table contains Soundfile and Hotsheet flags
(if these options have been exercised by the client).
Soundfiles - a green button with a music note indicates a sound recording is available. To listen, simply click on the green button. Files are streamed in MP3 and are available to listen to on-line or download. If you are experiencing problems, the recomended solution is to download the latest version of Internet Explorer and Windows Media Player.
Hotsheets - a hotsheet appears as a large amber 'Go' button in the 'Show' column and is activated by
pre-determined responses to particular questions. The objective of this function is to highlight cases which require immediate attention. For example, if the respondent is dissatisfied with a particluar aspect of the service, the product DOA or customer loyalty is low. Hotsheet profiles may be changed to align with on-going projects or ad-hoc initiatives. Cases that are not hotsheets have blue 'Go' buttons.
The scatter plot function enables users to analyse a variable by two performance indicators.
To do this you must have defined a subset of data for analysis. For example, you could plot Product Groups by Overall
Satisfaction, and Satisfaction with Keeeping the Customer Informed:
Select X-axis attribute: Select the attribute you wish to plot as the x co-ordinate
Select y-axis attribute: Select the attribute you wish to plot as the y co-ordinate
Select variable: Select the variable you wish to analyse
You have the option to select number of completed surveys on the x-axis as an alternative to an attribute,
providing an indication of the proportion of customers
being affected by the performance of any particular group.
A scatter plot is displayed with axis labelled by attribute. Each component has a point and label on the graph. The
appended table shows the component label, and number of surveys, and the two scores associated with each component.
The scatter plot function also has full drill down functionality.
Case ID - search on the client Case ID or unique identifier
that aligns with your internal systems.
Answers - apply filters to define a subset of data. Select a variable from the dropdown menu.
Values associated with the variable will appear in another dropdown box to the right or below.
Open text search - Search for a string of text within a verbatim comment. This function will return all cases
that match this criteria with links to case data and sound files. Also use this function to select all surveys done by a
specific company or handles by a particular agent.
Download data - Through the search function or following drill down in either the scatter or trend
function users can download the data for further anaylsis. This is in the form of a MS Excel spreadsheet and is sent to the email address associated with the
logon.
Listen to an audio recording of an interview. This effectively turns a quantitative piece of
research into qualitiative understanding of respondent issues. Research becomes irrefutable by putting comments into context
and auditing the quality of the researcher technique.
Individual interview results are sent to the appropriate people who are responsible for taking quick action.
An example of this is customers who are dissatisfied or who have requested further information.
Hotsheets are sent in a spreadsheet daily and are also available through Dynamic Reporting in real-time.
Through the search function or following drill down in either the scatter or trend function users can download the
data for further anaylsis. This is in the form of a MS Excel spreadsheet and is sent to the email address associated with the
logon.
Methodology
Sample sizeHow many surveys are needed for the results to be valid?
We are often asked how results from a relatively small number of people can speak for everyone. Market researchers often
explain "sampling" to non-researchers by likening it to a cake. You only have to try a bite to know what it
tastes like. The same is true for public opinion. You don't have to ask every single person in your target
population what they think; you only have to ask a sample to get the flavour of their opinion.
This begs the question; how big does a sample need to be?
The sample size required is dependent on the amount of variability
in population response/opinion as well as the precision level required in the results. Formulae can be applied to determine the
required sample size for results with a given confidence level and similarly, once results have been collected,
confidence in results can be determined for a given sample. Typically,the more heterogeneous the population, the greater the
sample size required to meet the same level of precision.
Generally, the greater the sample size, the more confident we can be in results, but ultimately there will become
a point at which the sample is such a size that there is little to be gained in terms of greater accuracy of results
by increasing the sample size.
The following graph illustrates a critical number of surveys (n) above which additional sampling will
provide little extra value in terms of confidence.
Given that the sample mean is an estimation of the true population mean, there is undoubtedly a margin of error.
Specifying a confidence interval determines an error bound on the sample mean, for example, a 95% confidence level
indicates that we can be confident that the true population mean will be included in the specified interval 95 times out
of 100. The interval takes the following form:
x-E < u < x+E
where u is the true mean, x is the sample mean and E is the margin of error which is calculated based on the confidence
required, the sample variance and the sample size. (Assumes the sample is large enough to be considered 'normal' distribution)
The magnitude of the margin of error, and hence the width of the interval, depends on the precision with which you wish to
report confidence in the results. The greater the confidence level required, the wider the confidence interval will be.
For further explanation see Margin of Error.
How can I be sure the score for my team is correct?
The margin of error provides a boundary of error associated with the sample mean. For example, if a sample size of 400 has a margin of error (E) of plus or minus 3 percent at a
95% level of confidence, this means that 95 out of 100 respondents' results in the sample will not be greater or
less than 3 percentage points off the actual mean that would have been obtained by interviewing all qualified
respondents in the target population.
As a rule, the lower the margin of error the more accurately the views of those surveyed matches those views of the
entire population. Margin of error (E) is calculated using the following formula where z is the figure associated
with 95% confidence level
How can I be sure the score for my team is correct?
A sample size of 400 has a margin of error of plus or minus 7 percent at a 95% level of confidence.
This means that 95 out of 100 respondents' results in the sample will not be greater or less than 7
percentage points of what would be obtained by interviewing all qualified respondents. As a rule, the lower the
margin of error the more accurately the views of those surveyed matches those views of the entire population.
Sample bias
Do the opinions of the people that respond represent the opinions of all my customers?
In short, the true opinions of an entire group, or score for a group, can only ever be estimated and
the reality is that subgroups of people may have a greater inclination to respond to a survey that others. This can be
for a number of reasons e.g.
Research methodology: Due to sample bias, the chosen methodology can largely influence the
scores from respondents. On-line studies are more vulnerable to sample bias than telephone as it is easier to
close down a pop-up survey or an email invitation than to say no to an interviewer on the other end of a telephone.
Representation: If you commission an on-line survey, only those with web access can complete a survey. You
cannot always guarantee that this group will represent the views of, for example, all 25-30 year olds in a population.
Motivation to respond: Respondents are more likely to respond if they have a particular motivation to do so. Polarization is one effect. Respondents who have received good or bad service in excess of expectations are more likely to respond, and will account for a disproportionate volume in a sample. DOA, or a problem not resolved are
also motivations to respond due to the hope that feedback will initiate a response.
Non-sample biasCan I expect different scores if I do surveys over the web?
In addition to sample bias, performance measures are prone to non-sample bias. This can be illustrated by web versus telephone research. Studies have shown that people do not like to appear negative in the
presence of others, and, inevitably, responses over the telehone tend to be higher than completing an on-line survey where
there is no human interaction. The result is a downwards shift.
Reliability & validityHow accurate is the survey tool in measuring performance?
Research is futile unless it is both valid and reliable:
validity - is the research tool accurate in serving the purpose for which it is designed and does it provide
correct information? For example, asking :
"How satisfied were
you with the time to reach an agent?" leaves an element of ambiguity in the question, which could be interpreted differently by different respondents,and hence scores given would not all be valid.
Does this question refer to an agent who dealt with the case, a switchboard, does it incorporate the IVR portion?
Questionnaire design and wording are vital in achieving the objective of valid results.
reliability - is the research tool consistent over time? Don't despair! Although all research is vulnerable to
unquantifiable degrees of sample and non-sample bias, the objective is consistency over time. Provided the survey
tool remains consistent, this research methodology will fullfil the objective of tracking change over time.
The impact of process change, training or improvement initiatives will be reflected in key performance
indicators.
*A very important point to consider is the danger in combining research methodologies. On-line and telephone
survey methodologies are reliable in their own right but employed simultaneously they render results unreliable.
Rememeber that these methodologies are subject to varying degrees of sample and non-sample bias and have
fundamentally different distributions, means or proportions, and variance. It is dangerous drawing conclusions
as to whether a change in score is the result of a genuine shift in performance or a change in the proportions
of web and telephone surveys.
The questionnaire will show you a variety of questionnaire styles and get your feedback on our site, as well as general information about your thoughts on ISO and research services. Surveys can be branded and routed to the voyage of the individual customer. Please click on the Start button to complete the demonstration.
