Loading

Instructions

A quick guide on how to use the search engine

Synthometricon (from Ancient Greek synthetikos "synthetic", metrikos "pertaining to measurement", and -kon from lexikon, together "dictionary of measured synthetics") is a search engine designed to help researchers, practitioners, and students discover scales from a database of more than 28,000 questionnaires, surveys, and tests.

Two main modes of operation are currently supported:

Item-based search. Enter multiple survey items (i.e., statements or questions) to find related instruments. This is the preferred search mode, using a fine-tuned model to predict item-pair correlations (magnolia-psychometrics/surveybot3000) for higher accuracy.
Label-based search. Describe a construct, trait, or scale name to find relevant instruments. This mode uses a general-purpose model and yields results that may be more prone to jingle-jangle fallacies (sentence-transformers/all-mpnet-base-v2).

Motivation

Behavioral science faces a proliferation problem. Thousands of psychological measures exist, but researchers often struggle to determine whether their "new" construct is genuinely novel or merely reinvents existing measures under different labels. This problem arises from the inherent asymmetry that creating new measures is easy and reputationally rewarding, while systematically comparing them against thousands of existing ones is cumbersome and time-consuming. Traditional validation methods simply don't scale to the vast landscape of published measures. This search engine addresses this gap by enabling researchers to quickly identify conceptually similar measures, helping to reduce redundancy, improve construct clarity, and ultimately make psychological science more cumulative and efficient.

While earlier approaches using latent semantic analysis have shown promise (Rosenbusch et al., 2020), this work uses specialized state-of-the-art transformer models and a substantially larger dataset to better predict the empirical relationships between psychological measures.

FAQ

What are the data sources for the search engine?

The data are primarily sourced from the APA PsycTests Database with permission from the American Psychological Association (APA). The database holds more than 28,000 instruments and 71,000 individual scales and subscales, built from more than 571,500 items that were extracted by parsing PDFs using (visual) language models. A handful of instruments come from other sources, which are labelled as such in the results.

Instruments are hierarchical. A scale may contain subscales, which may themselves contain subscales, up to four levels deep. Every level gets its own vector, pooled from all the items beneath it, so an instrument's overall representation and each of its parts can be searched and compared independently.

How trustworthy and accurate are the results?

Synthometricon estimates how strongly your items would correlate with each scale in the database if both were given to the same people. We checked these estimates against real data in two studies. In the first, with 22 well-known personality and well-being questionnaires (493 participants), the estimates correlated .87 with the observed correlations between scales. In the second, with 57 scales drawn to cover the variety in the PsycTests database (387 participants), they correlated .83. A typical estimate was off by about .12 in the first study and .16 in the second. The first study is a best case; the second is closer to what you can expect for the database as a whole.

Will I find scales that measure something very similar to mine?

Of 10 scales that truly correlate .70 or more with yours, about 9 appeared with a displayed value of .30 or higher in the second study, and about 7 at .40 or higher. In the first study, all of them reached .40. Of scales that correlate .50 or more, about 7 in 10 appeared at .30 or higher in the second study. This means: If nothing in your results lies above .30, a strongly overlapping measure is unlikely, but not ruled out.

How far down the list should I read?

To see 9 in 10 of the scales that truly correlate .70 or more with yours, read down to a displayed value of about .30. For scales that correlate .50 or more, the same share requires reading down to about .17. For well-known personality measures, .55 and .30 were enough. The engine shows how many results lie above any value, and the filter box helps narrow long lists.

Does it matter how many items I enter?

Yes, you will get better results with three or more items.

What are warning codes and what do they mean?

Warning codes indicate potential data quality issues detected during automated AI-extraction from source PDFs. Most cases flagged by these errors are not problematic, but highlighted records may require additional scrutiny. Regardless of warning codes, we recommend verifying any critical information against the original source when possible. Four checks can raise a warning: The extracted item count or scale count differs from the instrument's catalogue record, the extracted item text may not be a verbatim quote of the source PDF, or one or more items were translated into English for this corpus.

What "Could not be verified" means. Some checks cannot run at all. An instrument outside the APA PsycTests catalogue has no record to compare counts against, and a PDF with a thin text layer cannot be checked for verbatim item text. Those cases are reported separately from checks that ran and found nothing, because treating the two as the same would quietly present unverifiable rows as verified ones.

Separately, a result may note that some items point against the keyed average of the rest of their scale. This information is infered by the embedding model and may also flag records that require caution. However, the search results assume that that the information provided in the PDF document is correct.

In summary, warning flags are overly sensitive by design.

What does the similarity score mean?

In item-based search, cosine similarity scores are closely aligned with the correlation coefficients one would expect when administering two scales to human respondents. In label-based search, similarity scores reflect semantic relatedness between the search term and the scale and/or instrument labels in the database.

While cosine similarity theoretically ranges from -1 to 1, sentence vectors rarely yield negative scores.

Expanding a result shows a score for the instrument as a whole and one for every scale and subscale inside it. Each is computed against that level's own pooled representation, so a single subscale can match your query more strongly than the instrument it belongs to. The instrument pools every item in the document, including the ones your query has nothing to do with. The headline score of a result is the strongest match anywhere in its tree.

How can I access the full scales and items? Can I download or export search results?

Unfortunately, due to copyright restrictions, we cannot display individual survey items (statements/questions) directly on the platform. However, you can follow the DOI link to obtain the items from the original source. To access full instrument details, please:

  • Click the DOI link in search results to access the original source
  • Contact the instrument authors for permission to use the full measure
  • Check institutional access through your library's APA PsycTests subscription

Found a bug in the app or an error in the data?

Please let us know by opening an issue in this GitHub repository or by contacting us via email.

Research & Citation

More on the technical background

Search engine

The Synthometricon uses a specialized embedding model to predict the relatedness of survey items and scales. The database currently includes about 30,000 scale representations obtained from embedding statements from approx. 500,000 items. It is designed to facilitate the discovery of relevant measures in the behavioral sciences.

For the main application, please cite the following work:

Hommel, B. E., Külpmann, A. I., & Arslan, R. C. (2026). Synthometricon: A search engine to identify conceptual overlap in measures in the behavioral sciences. Manuscript in preparation.

@UNPUBLISHED{HommelKuelpmannArslan2026,
    author = {Hommel, Bj{\"o}rn E. and K{\"u}lpmann, Annika I. and Arslan, Ruben C.},
    title = "{Synthometricon}: A search engine to identify conceptual overlap in measures in the behavioral sciences",
    note = "Manuscript in preparation",
    year = "2026"
}

Embedding model

The conceptual search functionality is powered by a specialized language model trained to predict relationships between psychological items and scales:

Hommel, B. E. & Arslan, R. C. (2025). Language Models Accurately Infer Correlations Between Psychological Items and Scales From Text Alone. Advances in Methods and Practices in Psychological Science, 8(4), 25152459251377093. doi:10.1177/25152459251377093

@article{HommelArslan2025,
    author = "Hommel, Björn E. and Arslan, Ruben C.",
    title = "Language {{Models Accurately Infer Correlations Between Psychological Items}} and {{Scales From Text Alone}}",
    date = "2025-10",
    journal = "Advances in Methods and Practices in Psychological Science",
    year = "2025",
    journaltitle = "Advances in Methods and Practices in Psychological Science",
    shortjournal = "Adv. Methods Pract. Psychol. Sci.",
    volume = "8",
    number = "4",
    pages = "25152459251377093",
    issn = "2515-2459, 2515-2467",
    doi = "10.1177/25152459251377093",
    urldate = "2025-12-18",
    abstract = "Many behavioral scientists do not agree on core constructs and how they should be measured. Different literatures measure related constructs, but the connections are not always obvious to readers and meta-analysts. Many measures in behavioral science are based on agreement with survey items. Because these items are sentences, computerized language models can make connections between disparate measures and constructs and help researchers regain an overview over the rapidly growing, fragmented literature. Our fine-tuned language model, the SurveyBot3000, accurately predicts the correlations between survey items, the reliability of aggregated measurement scales, and intercorrelations between scales from item positions in semantic vector space. We measured the model’s performance as the convergence between its synthetic model estimates and empirical coefficients observed in human data. In our pilot study, the out-of-sample accuracy was .71 for item correlations, .89 for reliabilities, and .89 for scale correlations. In our preregistered validation study using novel items, the out-of-sample accuracy was slightly reduced to .59 for item correlations, .84 for reliabilities, and .84 for scale correlations. The synthetic item correlations showed an average prediction error of .17, and there were larger errors for middling correlations. Predictions exhibited generalizability beyond the training data and across various domains, with some variability in accuracy. Our work shows language models can reliably predict psychometric relationships between survey items, enabling researchers to evaluate new measures against existing scales, reduce redundancy in measurement, and work toward a more unified behavioral-science taxonomy.",
    langid = "english"
}

About

A meta-scientific project for better research practices.

This research is part of the SYNTH research project (#546323839), which is conducted as part of the META-REP Priority Program aiming to improve the replicability, reproducibility, and generalizability of empirical research in the behavioral sciences. SYNTH contributes to these goals by integrating large language models into current research workflows to reduce burdens on scientists while improving transparency and replicability.

Authors

Björn E. Hommel

Björn E. Hommel

Postdoctoral Researcher

Leipzig University

Annika I. Külpmann

Annika I. Külpmann

Doctoral Student

Witten University

Ruben C. Arslan

Ruben C. Arslan

Professor, Psychological Research Methods

Witten University

Corresponding author

Björn E. Hommel

Postdoctoral Researcher

Department of Personality Psychology and Psychological Assessment

Wilhelm Wundt Institute of Psychology Leipzig University

bjoern.hommel@uni-leipzig.de

Acknowledgements

We gratefully acknowledge the German Research Foundation (Deutsche Forschungsgemeinschaft, DFG) for funding this work (#546323839), and the American Psychological Association (APA) for granting permission to use the APA PsycTests Database for this research.

This web application is a research preview maintained by magnolia psychometrics GmbH.