Abstract: The impressive recent performance of DNN-based systems can be partially attributed to high-quality datasets -- indeed -- often, multiple at once. However, the effects of the datasets, especially how they interact with each other, are not studied well. We propose a state-vector framework to quantify the effect of the data itself. This framework uses idealized probing task results as the bases of the vector space and allows us to quantify the individual and interaction effects of datasets. We show that the significant effects of some commonly-used language understanding datasets are characteristic, and are concentrated on a few linguistic dimensions. Additionally, we observe some ``spill-over'' effects: the datasets could impact the models along dimensions that might appear irrelevant to the intended tasks. Our state-vector framework provides a systematic approach to the study of the effects of data, a crucial component in responsible model development.