Tips for a Successful Grow Trial Posted on April 8, 2022 | by Jeff Hayward Our industry has always been at the cutting edge of horticulture and controlled-environment growing. This is due to a complex set of factors. But I think one major reason that our industry remains at the forefront of innovation is due to a unique culture among growers. Growers in our industry tend to be early-adopters – always experimenting with new tools and techniques. Always looking for the next big thing. But how can we assess the effectiveness of a new piece of equipment, or a new product, or a new technique? How can we know if the ambitious claims being made by sales and marketing people are accurate? How can we be sure that the change will work in our operation, or that it will provide an acceptable return on investment? The only way to be confident in the effectiveness of any new growing practice is by testing it out in real-world conditions. This article is meant to provide some guidelines on how to conduct a high-quality side-by-side trial within practical limitations. Average growers do not need to conduct trials with the rigour characteristic of peer-reviewed scientific research. After all, I’m not a scientist. I just play one on YouTube. Nonetheless, following these guidelines may help you to cut through the hype and judge the latest technologies and techniques for yourself. It may not be possible or even desirable to precisely follow all of these guidelines when setting up a grow trial. But this discussion is mean to help to provide some direction to would-be grow-testers. Guidelines The Hypothesis At the risk of sounding like your least-favourite high school science teacher, I want to begin with the hypothesis… I know, this part feels super lame, but before getting started with a grow trial, it’s a good idea to think about what we’re trying to learn and how we will assess the results. The best way to do this is by creating a hypothesis. For example; “Product X will lead to a yield increase at harvest” or “Technique Y will increase the percentage of viable clones during propagation”. This statement will be proven or disproved by conducting the trial, and it can be a useful step to help clarify exactly what we are trying to achieve. Assessment Criteria How will we assess the effectiveness of the treatment? It is important to establish the methodology before we begin, and to stick to it as we proceed through the trial. For example, we may weigh the crop at harvest or after drying and curing. Or we may allow a set number of days for clones to develop roots, and then count the number of rooted clones after that time period. There are, of course, more subjective assessments that any grower will make (“the plants just look healthier”), and these are also valid. But we should look for ways that our results can be quantified whenever possible. Treatment and Control It is very common for growers to adopt a new technique or incorporate a new product by applying it to every plant in the garden all at once. When we do this, we rely on our growing experience to validate the results. The problem with this approach is that it makes it nearly impossible to determine where the results are coming from. Was the crop better than average because something new was added, or would it have been a banger crop anyway? Or if the results weren’t so good, how can we be sure that we didn’t mess up somewhere along the way? Maybe the results would have been even worse without the treatment. The only way to determine the effects of a new addition to the garden is to divide the crop into two groups. The “treatment” group will receive the new product or technique, while the “control” group does not. This will provide a baseline to accurately assess the results from the new addition. Include the Largest Sample Size Possible Without getting into a discussion of the statistical concept of p-value, it is important to recognize that the size of a trial matters when it comes to the quality of the data being generated. Technically, we can perform a side-by-side trial with only two plants; one in the treatment group and one in the control group. But this approach would yield very low-quality data, since we could never be sure if the effects we observe are due to the treatment, or just some statistical anomaly. Everything else being equal, the quality of the data will be proportional to the size of the trial. In others words, the results from a very small trial will not be nearly as reliable as those from a much larger trial. So, we need to perform as large a trial as is practical. For hobbyists and tent growers, smaller trials will be necessary, but they can still provide useful data provided that the grower recognizes the limitations. For large-scale and commercial growers on the other hand, it is not necessary or advisable to match the size of the treatment group to that of the control. If there are hundreds or thousands of plants under cultivation, we may choose to run one row, or one table, or a few lights as the treatment group against the rest of the crop. In general, a few dozen “treatment” plants will be sufficient to generate a reasonably good data set. And this more limited approach is wise as it will help to mitigate risk, since we may see poorer results in the treatment group than the rest of the crop. Minimize the Variables In order to separate the signal from the noise, we need to ensure that the variable being tested is the only difference between the “treatment” and “control” group. This means that we want to use the same genetics from the same batch of clones, planted at the same time in the same growing environment, with all of the same environmental parameters. The only difference between the two groups should be the treatment being tested. This way, whatever effect we see in the results can be safely attributed to the treatment. It is also important to consider how micro-climates might affect a trial. It is best not to put all of the treatment plants at the end of a row, in a corner or near the outside edge of the growing area. We can expect more representative results by locating the treatment group in an area where conditions are most “average”. In a full-on scientific research experiment we would scatter the treatment plants at random throughout the growing area to eliminate the possibility that any differences within the growing environment would hamper our results. Although this is certainly best practice, such randomization presents a level of inconvenience that would make it an impractical approach for most growers. Clearly Mark the “Control” and “Treatment” Groups This may seem obvious, but it is imperative that the plants in each group are clearly marked throughout the trial. This way, we avoid applying the wrong treatment to the wrong group and voiding the results. I’m ashamed to admit it, but this is a lesson that I’ve had to learn the hard way. Twice. Stick to the Plan As the crop progresses, we may begin to see some differences between our two groups of plants. In this situation, it can be tempting to go in and try to fix the side that is not performing as well. This is especially true for growers who have spent their careers carefully watching for and correcting any issues they may find in their gardens. However, it is important to let the trial run its course. If we have done everything else right, then we should be fairly confident that any differences we observe are, in fact, due to the variable we are testing. So, trust the process, and resist the impulse to intervene to make the under-performing plants catch up to the stronger plants. Conduct Multiple Replications At this point, we have carefully conducted our grow trial and the results seem promising. Now, depending on the scale of the trial, it may be a good idea to repeat the process in order to verify that the results are accurate. For hobbyists growing at a smaller scale, this may be especially useful since smaller trials provide a lower quality of data (as we discussed above). By replicating the results, we can increase our confidence that the results are indeed accurate. The time scale of the trial is also a factor here. It is much more practical to repeat a trial on, say, propagation than to repeat a full crop cycle trial. Cloning only requires a week or two and relatively little labour and space, so running multiple replications is a fairly simple prospect. Trials that take place over the full crop cycle may represent a commitment of three to six months, or an entire growing season in the case of outdoor crops. This time scale also involves a much larger commitment of labour and other resources, which may make additional replications impractical. And if the first replication involved a large number of plants, we may already have a high degree of confidence in the results which would make additional replications unnecessary. It is up to the grower to decide how many, if any, additional replications are appropriate in each situation. Just for context, field trials conducted in the world of conventional agriculture are generally required to include replications over three full growing seasons before they are taken seriously. Obviously, this approach provides excellent data, but it may also be one of the factors that explains why conventional farming can be a bit slower to adopt new techniques and technologies than their hydroponic counterparts. Trust the Data Hopefully this guide helps to get you thinking about how to effectively assess new products and techniques in the garden. Again, we are not trying to conduct university-level science here, and we may not be able to follow each and every one of these guidelines perfectly. But if we do our best to implement as many of these suggestions as possible, then we should do a good job of separating fact from fiction in the garden, allowing us to identify which treatments are truly useful and which are merely snake oil. Jeff Hayward – MIIM Horticulture Ltd.