Want to know the secret to always running successful tests?
The answer is to formulate a hypothesis.
Now when I say it’s always successful, I’m not talking about always increasing your Key Performance Indicator (KPI). You can “lose” a test, but still be successful.
That sounds like an oxymoron, but it’s not. If you set up your test strategically, even if the test decreases your KPI, you gain a learning, which is a success! And, if you win, you simultaneously achieve a lift and a learning. Double win!
The way you ensure you have a strategic test that will produce a learning is by centering it around a strong hypothesis.
So, what is a hypothesis?
By definition, a hypothesis is a proposed statement made on the basis of limited evidence that can be proved or disproved and is used as a starting point for further investigation.
Let’s break that down:
It is a proposed statement.
- A hypothesis is not fact, and should not be argued as right or wrong until it is tested and proven one way or the other.
It is made on the basis of limited (but hopefully some) evidence.
- Your hypothesis should be informed by as much knowledge as you have. This should include data that you have gathered, any research you have done, and the analysis of the current problems you have performed.
It can be proved or disproved.
- A hypothesis pretty much says, “I think by making this change, it will cause this effect.” So, based on your results, you should be able to say “this is true” or “this is false.”
It is used as a starting point for further investigation.
- The key word here is starting point. Your hypothesis should be formed and agreed upon before you make any wireframes or designs as it is what guides the design of your test. It helps you focus on what elements to change, how to change them, and which to leave alone.
How do I write a hypothesis?
The structure of your basic hypothesis follows a CHANGE: EFFECT framework.
While this is a truly scientific and testable template, it is very open-ended. Even though this hypothesis, “Changing an English headline into a Spanish headline will increase clickthrough rate,” is perfectly valid and testable, if your visitors are English-speaking, it probably doesn’t make much sense.
So now the question is …
How do I write a GOOD hypothesis?
To quote my boss Tony Doty, “This isn’t Mad Libs.”
We can’t just start plugging in nouns and verbs and conclude that we have a good hypothesis. Your hypothesis needs to be backed by a strategy. And, your strategy needs to be rooted in a solution to a problem.
So, a more complete version of the above template would be something like this:
In order to have a good hypothesis, you don’t necessarily have to follow this exact sentence structure, as long as it is centered around three main things:
- Presumed problem
- Proposed solution
- Anticipated result
After you’ve completed your analysis and research, identify the problem that you will address. While we need to be very clear about what we think the problem is, you should leave it out of the hypothesis since it is harder to prove or disprove. You may want to come up with both a problem statement and a hypothesis.
Problem Statement: “The lead generation form is too long, causing unnecessary friction.”
Hypothesis: “By changing the amount of form fields from 20 to 10, we will increase number of leads.”
When you are thinking about the solution you want to implement, you need to think about the psychology of the customer. What psychological impact is your proposed problem causing in the mind of the customer?
For example, if your proposed problem is “There is a lack of clarity in the sign-up process,” the psychological impact may be that the user is confused.
Now think about what solution is going to address the problem in the customer’s mind. If they are confused, we need to explain something better, or provide them with more information. For this example, we will say our proposed solution is to “Add a progress bar to the sign-up process.” This leads straight into the anticipated result.
If we reduce the confusion in the visitor’s mind (psychological impact) by adding the progress bar, what do we foresee to be the result? We are anticipating that it would be more people completing the sign-up process. Your proposed solution and your KPI need to be directly correlated.
Note: Some people will include the psychological impact in their hypothesis. This isn’t necessarily wrong, but we do have to be careful with assumptions. If we say that the effect will be “Reduced confusion and therefore increase in conversion rate,” we are assuming the reduced confusion is what made the impact. While this may be correct, it is not measureable and it is hard to prove or disprove.
To summarize, your hypothesis should follow a structure of: “If I change this, it will have this effect,” but should always be informed by an analysis of the problems and rooted in the solution you deemed appropriate.
A/B Testing 101: How to get real results from optimization
The True Value of Data
15 Years of Marketing Research in 11 Minutes
Marketing Analytics: 6 simple steps for interpreting your data
Website A/B Testing: 4 tips to beat an unbeatable landing page
In the strict sense of the word a loop space in topology for a given pointed topological space is the mapping space (with its compact-open topology, se the example there) of continuous functions from the circle to , such that they take the given basepoint of the circle to the prescribed basepoint in (or if one drops this condition, then one speaks of the free loop space). One such continuous function may be thought of as a continuous loop in , and hence the mapping space is the space of all these loops.
If here is equiiped with further structure, such as smooth structure (e.g. a smooth manifold), then one may good cases find such extra structure also on the loop space, for instance to form a smooth loop space, etc. See at manifolds of mapping spaces for more on this.
If one regards this construction not in point-set topology but in classical homotopy theory of topological spaces (equivalently∞Grpd), then, up to weak homotopy equivalence, the loop space is equivalence the homotopy fiber product of the basepoint inclusion along itself.
Let Top be a nice category of topological spaces, in particular one which is complete, cocomplete, and cartesian closed. Let be the circle, i.e., 1-dimensional sphere, with chosen basepoint, and let be a space with a chosen basepoint. Then the loop space of (at ) is an internal hom
in the category of based spaces. Explicitly, it is given by the pullback in
(using exponentials to denote internal homs in ), in other words the function space of basepoint-preserving maps , whose basepoint is the constant map at the basepoint of .
The category is symmetric monoidal closed; its monoidal product is called the smash product, often denoted . In particular, the loop space functor
has a left adjoint obtained by taking smash product with . This left adjoint is called the suspension functor. Explicitly, the suspension is formed as the pushout
with basepoint provided by the right vertical arrow.
A loop space is an example of a A-∞ space, in particular it is an H-space. Loop spaces admit this rich algebraic structure which arises from the fact that the based space carries a correspondingly rich co-algebraic structure, starting from the fact that the based space is an H-cogroup.
The description of this structure on loop spaces has been the very motivation for the introduction of the notion of operad and algebra over an operad in (May).
An important theoretical consideration is when an H-space, and particularly one having the type of a CW-complex, has the homotopy type of a loop space of another CW-complex: . In this circumstance, one calls a delooping of ; an important example is where carries a topological group structure , and is the classifying space of .
The most basic fact about deloopings is the shift in homotopy groups:
which follows straight from the adjunction plus the fact that the suspension of is . (This isomorphism needs to be developed at greater length.)
The modern study of the question “when can an H-space be delooped?” was inaugurated by Jim Stasheff. The basic theorem is as follows (all spaces assumed to be CW-complexes):
This is due to (Stasheff). The analogous statement holds true in every (∞,1)-topos other than Top. Details on this more general statement are at loop space object and at groupoid object in an (∞,1)-category.
Local homotopy properties
Let the space be locally 0-connected and semi-locally 1-connected (i.e. it admits a universal covering space). The loop space for any basepoint is locally path connected, as is the free loop space . If is locally 1-connected and admits a basis of open sets such that is the zero map, then is locally 0-connected and semi-locally 1-connected, and so admits a universal covering space.
In general, if is locally -connected, is locally -connected. This process can obviously be iterated up to times, so that is locally 0-connected. This can be weakened to locally -connected and semi-locally -connected: this is just like the case but replacing with (as was done in the previous paragraph with ). We will actually define a space to be semi-locally -connected to include the condition that it is locally -connected. This result was proved for more general mapping spaces and various subspaces when is Hausdorff and a finite polyhedron in (Wada) but a much simpler and direct proof for general and or is possible.
The fundamental -groupoid of a space (Trimblean for choice) can be topologised to be an internal -groupoid in when is semi-locally -connected. Furthermore, the homotopy groups of the -groupoid, a priori topological groups, are discrete.
For , this is in David Roberts's thesis. For , it has been known for ages and is in Ronnie Brown's topology textbook.
There is a Quillen equivalence
between the model structure on simplicial groups and the model structure on reduced simplicial sets, thus exhibiting both of these as models for infinity-groups (Kan 58). Its left adjoint, the simplicial loop space construction, is a concrete model for the loop space construction with values in simplicial groups.
See also simplicial group and groupoid object in an (∞,1)-category for more details.
Daniel Kan, On homotopy theory and c.s.s. groups, Ann. of Math. 68 (1958), 38-53
Jim Stasheff, Homotopy associative -spaces I, II, Trans. Amer. Math. Soc. 108, 1963, 275-312
Peter May, The geometry of iterated loop spaces Lecture Notes in Mathematics 271 (1970) (pdf)
H. Wada, Local connectivity of mapping spaces, Duke Mathematical Journal, vol ? (1955) pp 419-425
The simplicial loop group functor is discussed in chapter V, section 5 of
See also the references at looping and delooping.
Revised on February 19, 2018 01:15:48 by Anonymous (220.127.116.11)