Before initiating the analysis, you have to determine the analysis population. The first question is, who will be included in that analysis? The gold standard for clinical trials is to include everyone according to their assigned intervention. This is often referred to as intention to treat or as randomized. The question you answer is, how does a participant assign to an intervention do? This mimics real care when a patient comes into you and says, "How should I be treated?" The pros of this are that it preserves the benefits of randomization. You keep your balance between your populations and your ability to analyze the data. It's unbiased in that it measures the effect in a global sense. If people are non-compliant, switch treatment, are misdiagnosed in a trial, that's probably going to happen in real-life. That real-life view is what we call effectiveness. There are some potential issues. The results may be influenced by a number of factors regardless of the fact that you are preserving randomization. This includes missing data, lack of compliance. Also for early trials, we may want to know more about efficacy than effectiveness. That is, how does the intervention do under ideal circumstances? What should we consider when we're thinking about exclusions from the analysis? Our main goal as always is to minimize bias. The biggest problems come when the exclusion criteria are related both to the intervention and the outcome. You should really use exclusion sparingly or as a sensitivity analysis. Despite your best efforts, you may still cause bias and lose the benefits of randomization. How do we minimize the risks? Well, it's important to have an objective determination of who and why to exclude someone. It should be equally applied to all arms to maintain comparable subsets. You have to be careful; this is not always possible. For example, if you're comparing placebo versus medication, you can use a blood test to measure adherence to the medication, but that's not possible with a placebo. It's important to base your criteria for exclusion on factors that are collected prior to the intervention, because this means they won't be influenced by the intervention. You do have to note the question that you answer has changed. You are now answering the question, how does a person who meets a certain set of criteria do on the intervention? Let's look at an example that highlights the importance of comparable subsets. This example is taken from the coronary drug project, which compared five treatment arms and the placebo arm. The primary outcome was five-year mortality in men, three months beyond a myocardial infarction. We're going to focus on the comparison of clofibrate and placebo. Overall, the mortality was very similar between the two groups, and this was a surprise since clofibrate was expected to reduce the mortality. The investigators were wondering if people who were actually adherent to clofibrate would do better. They looked at the following comparison. They looked at those who were good adherers and they compared that to all people taking placebo. Here you can see that the five-year mortality was lower for clofibrate than placebo. But there's a problem, they only compared good adherers versus everyone. Their assumption is that it doesn't matter how much placebo you take, so adherence shouldn't matter. Let's look at what really happened. If you compare the good adherers for clofibrate with placebo, then the five-year mortality was very similar. That's also the case for the poor adherers. What happened? Well, the fact that someone was a good adherer was not only indicating how much medication they got but also reflecting other factors about the participant. For example, they may be people who take better care of themselves, have more access to health care, and other things that may improve their survival. This shows us why it's so important that when you do have exclusions, they're applied equally to give you comparable subsets. What type of exclusions are considered? Remember, all have the potential for bias. One of the most common is eligibility criteria. Despite your best efforts, an individual may be enrolled when they're not eligible. Obviously, you need to treat them according to best medical judgment and not risk their health, but should they be included in the analysis? This has less risk if it's applied to both groups equally, but it could still cause problems with randomization. Another issue could be data quality. This could be individual items where a test was performed incorrectly, or it could be exclusions based on a site's performance where they in general performed tests incorrectly or have systemic errors or fraud. Of course, this also has other issues such as politics and collaborations. Missing data may be an issue. If you have no data on someone, how are they included? In some cases, statistical techniques can be used instead of an exclusion. Perhaps the most commonly used exclusion is treatment criteria. Some examples of criteria might be at least one dose, this is sometimes used for safety, met protocol standards, this is sometimes referred to as per protocol, achieved a predefined threshold such as 80 percent or more. However, there are real issues with using treatment criteria. It may be influenced by the intervention which could cause bias. You can't predict who's going to be compliant when they walk in the door. It's very hard to measure. Self-report can often be biased, and there aren't always other ways to capture the data accurately. There are a range of levels of compliance. It's not all or nothing, and it may fluctuate over time. For example, with more adherence at the beginning of the trial tapering off towards the end, and finally you do lose the protection of randomization. Adherence can be an important part of your analysis, but you must use the appropriate techniques. In this case, causal inference methods. Other post-intervention measures will also have similar problems to the treatment criteria.
