In four classes of materials—the layered copper oxides, organics, iron pnictides and heavy-fermion compounds—an unconventional superconducting state emerges as a magnetic transition is tuned towards absolute zero temperature, that is, towards a magnetic quantum critical point1 (QCP). In most materials, the QCP is accessed by chemical substitution or applied pressure. CeCoIn5 is one of the few materials that are ‘born’ as a quantum critical superconductor2, 3, 4 and, therefore, offers the opportunity to explore the consequences of chemical disorder. Cadmium-doped crystals of CeCoIn5 are a particularly interesting case where Cd substitution induces long-range magnetic order5, as in Zn-doped copper oxides6, 7. Applied pressure globally suppresses the Cd-induced magnetic order and restores bulk superconductivity. Here we show, however, that local magnetic correlations, whose spatial extent decreases with applied pressure, persist at the extrapolated QCP. The residual droplets of impurity-induced magnetic moments prevent the reappearance of conventional signatures of quantum criticality, but induce a heterogeneous electronic state. These discoveries show that spin droplets can be a source of electronic heterogeneity and emphasize the need for caution when interpreting the effects of tuning a correlated system by chemical substitution.