Before you send me a question, make sure you read this FAQ.

If you send me an email about one of your experiments or surveys, please include the following information:

  1. Your user account (in case it differs from the email you use)

  2. The experiment or survey name you write about

  3. The exact issue you want an answer on

Please do not send emails like the one’s below, they will not be answered:

  1. "Need BFI survey please" (click here to read why)

  2. "I am a student from …​ and need permission to use the scale …​" (click here to read why)

  3. "I have a 4x2x3 design and want to include both the STROOP and the WCST with autistic and typically developing participants. I have just seen your website. Would this work with PsyToolkit?" → Please try to find out the answer yourself first. Also, I am not a replacement supervisor, so do not ask complex questions about how to set up your survey (unless you are willing to make a donation and the question is clear and you are willing to read up yourself first).

Topics in this FAQ

General questions

How do I cite PsyToolkit?

Cite both papers:

  • Stoet, G. (2010). PsyToolkit - A software package for programming psychological experiments using Linux. Behavior Research Methods, 42(4), 1096-1104.

  • Stoet, G. (2017). PsyToolkit: A novel web-based method for running online questionnaires and reaction-time experiments. Teaching of Psychology, 44(1), 24-31.

Why is it called PsyToolkit?

Psy is short for Psychology and Toolkit refers to the various software tools PsyToolkit comes with.

What can one do with PsyToolkit?

A lot. Apart from this website with lessons and documentation, you can setup your own online or laboratory-based studies. In short, you can program and run cognitive psychological experiments, you can run surveys, and you can do this all offline or online, including online data collection.

I want to XYZ (replace XYZ with the description of your study), can I do this in PsyToolkit?

I am happy to answer such questions, but please first check the following link to the PsyToolkit documentation carefully, it most likely answers all your question: http://www.psytoolkit.org/lessons/#lessons4

Can I run a questionnaire and include an experiment as shown on the PsyToolkit website?

Yes, and it is reasonably easy to do. This tutorial explains best how to do it step by step.

Does PsyToolkit work with MTurk/Qualtrix/SONA (etc)?

Yes, there are various research groups doing so. It is very easy to do. PsyToolkit can also communicate with Mturk, Qualtrix, Sona using so-called CGI variables — if you want this, please contact the PsyToolkit developer for details, you will get an answer within one day. There is a fee to use this feature to support PsyToolkit (after all, you can perfectly well run online studies without these external websites, it is only for those who can already pay for expensive systems like Qualtrix/SONA, etc).

Are you on the social media?

All important PsyToolkit information is on its own page, so there is not that much need. But PsyToolkit is on various social media for documentation (YouTube) and news (twitter), and likes (Facebook). Here are the links:

How can I give credits back to the Sona system?

This is very simple. You can put a variable in the end URL. For example, you can use something like this (note, example link does not work):
https://my-university.sona-systems.com/webstudy_credit.aspx?experiment_id=1234&credit_token=12345abcdefghij123534003434343ez
Make sure you change the values for experiment_id and the credit_token

Why is this software free?

There is a lot of free software available, and there is a profound philosophy behind it. Read more about this here. Free software makes it more accessible to use, especially for students.

Where is PsyToolkit based?

The PsyToolkit webserver is based in a professional data center located in Strasbourg (mainland Europe).

Who pays for it?

The project depends on donations, for example to pay the data center. You can support by making a donation here.

Who uses PsyToolkit?

PsyToolkit is being used around the world by researchers and students. Many psychology students use it for their research projects.

What is the difference between Eprime, Superlab, and PsyToolkit?

There are many similarities, but also differences. PsyToolkit is on par with those packages. But unlike purely commercial packages, PsyToolkit aims to be educational and accessible, that is, there are many demos and surveys on the websites psychology students can learn from. You can run your whole non-commercial project without paying (but donations are welcome).

Does PsyToolkit work on tablets and mobile phones?

Yes, surveys can easily be run on mobile phones. Experiments too, but because experiments often use a keyboard, you need to decide whether you want to use key strokes in your experiment. Response times from touch screens are less precise than key presses.

Is the response measurement timing reliable?

It is different for the web based version and the Linux offline version. The latter is super precise. For many experiments, you probably do not necessarily need such high levels of precision. Yet, it is good enough to reliably measure relatively small effects, such as Inhibition Of Return or the Simon Effect (i.e., effects of less than 50 ms effects).

How can we support PsyToolkit?

You can do one of the following things:

  • Write about the software on the social media

  • Send me an email about what you like about the software, I am always interested to hear positive stories about people who benefit from the software, and it motivates me to expand the software.

  • Write a more formal letter of support to me explaining how you use the software and why you think it is good. Ideally, this letter would be sent as PDF. Such letters help me to get the support within my organisation.

  • Consider a donation.

  • Tell colleagues about the software.

  • Help with translating the survey interface into your language, if your language is not yet available.

Can you implement feature X, please?

I am always open to suggestions of new features. Just send me an email.

I have sent you a question, why did I not receive an answer yet?

I no longer reply to questions about permissions to use (see here why), because they are all answered in the FAQ. I do not answer to questions that are answered here in general. Sometimes, I am too busy and it may take time. I generally try to answer, but sometimes they get burried under the many newer emails I get. If you still have not found an answer and it has not been addressed in the FAQ, please try again.

How can I delete my account?

This is very simple. There are 2 completely different ways to do this:

  1. Just do not login for one year. If you do not login for more than one year, you will be reminded by email to login. If you do still not login, your account will be deleted.

  2. Go to settings, and select "delete your account". You then receive an email with a code you need to enter here to finish the process.

About the questionnaires on this web site (permissions, etc)

There are many questionnaires on PsyToolkit’s website. If you have a question about them, please make sure you read this first carefully. This because I get many of these questions, I will often not be able to answer them — that is exactly why I created this FAQ.

Replace the XYZ below with whatever scale you are asking about.
  1. I want to use one of your questionnaires. Can I get your permission to use it?

    I get this question quite a lot. Here are the rules:

    • Whenever you use PsyToolkit, you need to cite its two publications (see above)

    • Nobody is allowed to use PsyToolkit for commercial purposes or as part of a business, unless explicit permission has been given by me (this is rare).

    • Students can use PsyToolkit for educational or research purposes without further permission

    • The surveys in the survey library are almost all based on work by resarchers who have absolutely nothing to do with PsyToolkit. As PsyToolkit developer, I have personally checked that it appears (to the best of my knowledge, which is unfortunately not infallable) they can be used without violating copyright. In every online survey, there is a section about legal stuff. That is to the best of my knowledge. If you don’t trust it, please check yourself with the original study and its authors.

  2. I want a hard copy of XYZ’s scale, can you please send it because I need it for my dissertation?

    Because I do not own the scales, I cannot redistribute papers about them. You can typically find them in your library or sometimes even on the Internet.

  3. I want to use the XYZ scale with Google Forms/SurveyMonkey (etc), can you please tell me how I best do this?

    I can only answer questions about PsyToolkit.

  4. I do not get a response from XYZ via email, can you please help me instead?

    Some of the authors of the scales will either be retired or disseased. I am not in a position to answer on their behalve.

About online experiments

  1. After running my experiment, I can see the data, but how can I save it to my harddisk?

    It depends what exactly you want. There are different scenario:

    1. When you just want to program and test your experiment, you cannot directly "save" your data to harddisk. You need to copy and paste it. This is due to security mechanism of the browser.

    2. If you want to run an online experiment as part of online data collection, you can embed your experiment in an online survey. That way, the data will be saved on the PsyToolkit server, and you can download them from there. Check the following question.

  2. I have my experiment. But how do I get my experiment "online" so that people can "click" it?

    This is very simple. It takes a few steps:

    • First make sure your experiment on its own works well.

    • In addition to your experiment, you also must create an online questionnaire.

    • The online questionnaire needs to embed your experiment, that is very simple to set up (see example below).

    • Now compile your survey and then you get a URL which you can distribute to people.

    • Imagine you have an experiment called "stroop". Then you can embed in your survey as follows:

      l: myexperiment
      t: experiment
      - stroop
      You can read about this in detail about how to get your experiment online here.
  3. Is the scripting language for online experiments different from the Linux version?

    No, it is the same, although some features are designed only for the Linux version (such as external keyboards) and some features only for online experiments

  4. How can I collect data from multiple participants?

    This is very easy. You need to run the experiment within an online survey. You need to set up an online survey and embed your experiment in it. Then you get an online web link (URL) and you can give that to your participants. All the data will be collected online and can be analyzed within the PsyToolkit website. A complete lesson on how to do this can be found here.

  5. Which browser should I use?

    The online PsyToolkit version works fine with all modern web browsers. The PsyToolkit developer primarily works with Firefox, which is probably the best available browser.

Online questionnaires

  1. How long can surveys run?

    Currently, they can be active for as long as you want, but there are no guarantees.

  2. Can you support my language?

    Currently, PsyToolkit supports many foreign languages. Contact me if it does not support your language. It is relatively easy to add new languages, as long as someone is willing to translate the basic phrases used.

  3. Will surveys stay valid if the software is updated?

    Yes, the old versions will stay active. If you compile a survey with a specific PsyToolkit-for-the-web version, this will remain active, even though the latest version of PsyToolkit might have new or changed features. This makes the life of researchers easier.

  4. Which browsers are supported?

    All modern browsers, including Internet Explorer version 10 and higher. Embedding experiments with keyboard input requires a real keyboard (touchscreens won’t work very well or not at all).

  5. Is there a welcome screen for surveys?

    If you create a survey, you can put welcome-information in the various text boxes under survey intro screen.

  6. Can I upload multiple files at once?

    Yes.

  7. Where can I add information for participants (such as plain language statement, etc)?

    If you create a survey, you can put this information in the various text boxes under survey intro screen.

  8. How do I put my university logo on the welcome screen?

    Here are the two steps how to do this:

    1. Find the weblink to your logo, such as http://my_university.edu/logo.jpg

    2. In the "About this survey" section, enter <img src='http://my_university.edu/logo.jpg'>

    3. Save and compile and it should show up

  9. What sort of survey items are available?

    There are many different items available, including Likert scales, choice questions, text boxes, embedded experiments and videos, etc. Check the online documentation under Question Types. You can also copy and paste existing scales from the survey library.

  10. Can the order of questions be randomized?

    Yes (read how to)

  11. Can you randomly assign participants to questionnaires?

    Yes (read how to)

  12. Is there a tracker of how far the participant has come in a survey?

    At the top of the web page, the percentage completed is indicated. This might not always be accurate if you include conditional elements (i.e., you cannot predict how long the survey will last if certain questions might be skipped depending on how some questions are being answered).

  13. I have a "check" question, but I would like that people can enter some text if they want to choose something that is not on my list. Can I do that, and if so, how?

    Yes, you can do that, click here to see how.

  14. Can people go back with the browser back button?

    Yes

  15. How can you embed an experiment in a survey?

    You will first need to have a working experiment, and then you can embed it in the "experiment" item. An example can be viewed here.

  16. How can you embed a video in a survey?

    You can embed a YouTube video. Here is an example.

  17. I want that people can just skip a (radio) question. How do I do this?

    Just use the option free, as in the example below:

    Example
    l: example
    t: radio
    o: free
    q: What food do you like?
    - Potatoes
    - Rice
    - Bread
  18. What types of cognitive tasks are already available?

    There are a number of cognitive tasks in the online library. You will still need to understand data analysis techniques to extract the data you want. If there are special tasks you are interested in, you can program them yourselves, or make a suggestion to me to implement it.

  19. How can I invited participants to participate in the survey I have created?

    When you create a survey and select it to go online, a URL link is provided. You need to send this to your participants yourselves or advertise somewhere on a website, the PsyToolkit website will not do that for you.

  20. What sort of format will the data be stored in?

    The data will be available in a raw format (text file) and in the common spreadsheet format CSV, which can be opened with all spreadsheet software. Excel export is also supported.

  21. Can surveys be programmed so that participants get feedback about their own answers at the end?

    Yes (using the set question, click here to read about it).

  22. Can the text font of questions be changed?

    Yes, all text can be changed using html tags. Click here for some examples.

  23. Is there an option for "must fill out" questions?

    Yes, you can "require" that people give an answer. For "radio" (choice) questions this is the default, and for text boxes you can add the option require. Check the documentation about textlines on how this works.

  24. Can you have survey links that are unique to participants?

    Each survey you create has one survey link. This means, in essence, that participants are unidentifyable. You can, of course, ask people to identify them with a question about their email. Also, you can give people an identifier code at the end of the survey, which they can then use for course credits, etc. Check here how that works.

  25. Can you run surveys in different languages?

    Yes, the text on the buttons and so on can be set to quite many different languages.

  26. Can you change the button text?

    Yes, you can use the questionnaire item operator "b:", see Check here how that works.

  27. Can I download my data in SPSS format?

    Under download, you can specify that you want to add Excel files, which can be easily and directly read into SPSS.

  28. I have a survey with an embedded experiment. How do I download the mean response times and error rates of participants?

    You need to make sure that you specify the parameters of the embedded experiment and then download again. You can do this after data collection. Click the blue questionmark button at "Download participant data" for detailed instructions.

  29. Can I have multiple questions on "one" screen?

    No, that is currently not possible. It is rarely an issue, and the current system has advantages, because you can get the exact response time for each question answered. Even so, having multiple question items together is high on the todo list.

  30. Is there a way of knowing the response time of each question in the surveys?

    Yes. When you download your survey data, you will find a file named data_times. This file contains the response time for each question (in milliseconds).

  31. Which external website can I use to upload large sound or image files?

    You can use dropbox (create a dropbox account first). Basically, you want to do this as follows:

    1. Upload your file.

    2. Select "Share the file."

    3. In the share dialog, select "Copy the link"

    4. Now you can copy and paste the link to your PsyToolkit script

    5. You are not yet done yet! The link as given is not entirely correct, you need to make a small change: You must make absolutely sure that at the end of the link, you look for the question mark. Remove anything that follows in the link after the questionmark and make sure the URL ends with ?raw=1

      For example, look at this URL for a random sound file:

      Change it into (just ending different, removed ?dl=1 and changed that into ?raw=1)

    6. Now use the link that ends in raw=1 in your PsyToolkit script

Data storage

  1. Where are online data stored?

    Data are stored on the PsyToolkit webserver. This server is based in a Central European professional data center (Strasbourg).

  2. Who manages the server?

    The server is managed by Professor Gijsbert Stoet (PhD, Chartered Psychologist based in the UK).

  3. In your experience, have IRBs/Ethics Committees generally approved of PsyToolkit’s use, the security of the server more specifically?

    Yes, this has never been an issue.

  4. Can stored data be lost?

    This is very unlikely, because data are being backed up daily. If you delete a file on the server though, the backup will also delete it within 24 hours (so if you want it back, please contact me as soon as possible). PsyToolkit does not give any guarantees, though (see next point about what you can do to always be on the safe side).

  5. Can I store the data myself?

    Yes, you can download all your surveys, experiments, and data via the web interface. If you are concerned, just download your data yourself. As said, it is unlikely to get lost, but if your data is important to you, I recommend you to make sure that you make regular downloads. It is easy to do.

  6. Who has access to the data collected in online surveys?

    Only PsyToolkit developer Professor Dr. Gijsbert Stoet has access to all data, because he manages the PsyToolkit server.

  7. Is there any "no data loss" guarantee?

    Although data loss is extremely unlikely because of the distributed backup system (see above), you are responsible for your data. There are no guarantees given by PsyToolkit. Please note that you can download all collected data anytime to make a backup copy on your own computer at home. This is recommended.

  8. Are online survey participants informed about data storage?

    Online survey participants are informed about data storage at the bottom of the welcome survey of each online survey. It is possible for researchers not to show the welcome screen. The researchers need to choose how important this information is to them (there may be situations where the information is provided elsewhere or differently).

  9. Is the IP address or country information of participants stored?

    I sometimes get asked this because ethics review boards migth not like it if you store the IP address of a participant. Therefore, PsyToolkit gives you the option to save or not save the IP address of the participant. Please note the following as well: Some people overestimate the capacity to identify someone using an IP. In reality, IPs are not as useful as some people think they are. Even though IP addresses are linked to individual computers, it is typically practically impossible to identify who uses an IP. In theory, there will be law enforcement agencies who can, with time-consuming detective work, find out where a participant would be located, but the reality is that in practical terms, the IP address is still pretty anonymous and of little use except for finding out in which country the participant is. You can read more about this here.

  10. Does the server use secure transmission?

    Yes, the server uses the secure version of HTTP, the web protocol used for communication between computer browsers and servers. This means that all communications are encrypted. In many browsers, this is indicated with a "padlock" symbol in the addressline.

Writing experiments

Some Q&A about how to set up your own experiments.
  1. What is the output data format of experiments?

    The output data format is ascii text. Further, it is entirely up to the designer of the script which variables are being save (with the save statement).

  2. Can PsyToolkit play a sound stimulus, like a short beep?

    Yes it can. See the statment sound and link

  3. Can PsyToolkit play some music when a participant is doing a block of trials?

    Yes it can do that too. See the statment sound.

  4. Can I include files into my script?

    Yes, see the include statement.

  5. Can PsyToolkit use functions?

    There are no "functions" which can pass parameters, but you can use the "part" syntax to use the same code snippets at multiple pages. Read here about the part statement.

  6. Can experiment use "special" keys, like the arrow keys for responses?

    Yes, you can use all of them. Read here in online docs.

  7. In designing an experiment on your platform, I noticed that the blocks that call the table replace each trial that is called (i.e in one block it would be theoretically possible to get the same stimuli for all trials). Is there anyway to set it to where the block calls stimuli without replacement?

    In your tasklist statement, use the all_before_repeat option. Read the details about tasklist here.

Linux version

PsyToolkit experiments can be compiled on Linux computers. This is only of interest for some people. If you do not know what Linux is, please just skip this section altogether.
  1. Is there a recommended Linux distribution?

    No, PsyToolkit for Linux should run on any modern distribution. Mint Linux is one of the most popular. Linux is tested mostly on Fedora, Mint, and Ubuntu. The PsyToolkit developer works with all these 3 Linux distributions.

  2. Where can I download the PsyToolkit software package for Linux?

    See the Linux page of PsyToolkit for details how to get the install files.

  3. Will you implement this on other platforms, such as Mac or Windows?

    No, but there is other free software available for those platforms. Of course, you can also just run the web-based version on Windows or Mac.

  4. Which external keyboards does PsyToolkit support?

    Cedrus, BlackBox and IOlab (although these latter ones are probably no longer being made or sold). You can also build your own keyboard and use the parallelport ().

  5. Does PsyToolkit work with the new Wayland graphics stack?

    Yes. PsyToolkit uses the SDL (version 2) libraries for graphics and sound. SDL supports Wayland. Of course, Wayland is the default graphics stack on Fedora Linux, which the PsyToolkit developer works on most.