General questions

  1. How do I cite PsyToolkit?

    Cite both papers:

    • Stoet, G. (2010). PsyToolkit - A software package for programming psychological experiments using Linux. Behavior Research Methods, 42(4), 1096-1104.

    • Stoet, G. (2017). PsyToolkit: A novel web-based method for running online questionnaires and reaction-time experiments. Teaching of Psychology, 44(1), 24-31.

  2. Why is it called PsyToolkit?

    Psy is short for Psychology and Toolkit refers to the various software tools PsyToolkit comes with.

  3. What can one do with PsyToolkit?

    A lot. Apart from this website with lessons and documentation, you can setup your own online or laboratory-based studies. In short, you can program and run cognitive psychological experiments, you can run surveys, and you can do this all offline or online, including online data collection.

  4. Does PsyToolkit work with MTurk?

    Yes, there are various research groups doing so. It is very easy to do. You can tick the end-code box in PsyToolkit to get a unique participant code which participants then can enter in their MTurk entry box.

  5. Are you on the social media?

    All important PsyToolkit information is on its own page, so there is not that much need. But PsyToolkit is on various social media for documentation (YouTube) and news (twitter), and likes (Facebook). Here are the links:

  6. Why is this software free?

    There is a lot of free software available, and there is a profound philosophy behind it. Read more about this here. Free software makes it more accessible to use, especially for students.

  7. Where is PsyToolkit based?

    The PsyToolkit webserver is based in a professional data center located in Strasbourg (mainland Europe).

  8. Who pays for it?

    The project depends on donations, for example to pay the data center. You can support by making a donation here.

  9. Who uses PsyToolkit?

    PsyToolkit is being used around the world by researchers and students. Many psychology students use it for their research projects.

  10. What is the difference between Eprime, Superlab, and PsyToolkit?

    There are many differences, even though these software packages have many similar aims. One difference is that PsyToolkit does not have a graphical user interface for programming your experiments. Another difference is that PsyToolkit does not have the resources for full user support like commercial companies do. Otherwise, PsyToolkit is on par with those packages.

  11. Why not a graphical user interface?

    Although a graphical user interface might give you the impression it is easier to use, it is actually much easier and faster to work with a text-based setup.

  12. Does PsyToolkit work on tablets and mobile phones?

    Yes, but there are limitations for cognitive psychological experiments. Typically, you want to collect response times, and keyboards and mice are reasonably good input devices. With only a touch screen, the online surveys work fine, though (although you cannot do experiments with keyboard input).

  13. Is the response measurement timing reliable?

    It is different for the web based version and the Linux offline version. The latter is super precise. For many experiments, you probably do not necessarily need such high levels of precision. Yet, it is good enough to reliably measure relatively small effects, such as Inhibition Of Return or the Simon Effect (i.e., effects of less than 50 ms effects).

  14. How can we support PsyToolkit?

    You can do one of the following things:

    • Write about the software on the social media

    • Send me an email about what you like about the software, I am always interested to hear positive stories about people who benefit from the software, and it motivates me to expand the software.

    • Write a more formal letter of support to me explaining how you use the software and why you think it is good. Ideally, this letter would be sent as PDF. Such letters help me to get the support within my organisation.

    • Consider a donation.

    • Tell colleagues about the software.

    • Help with translating the survey interface into your language, if your language is not yet available.

  15. Can you implement feature X, please?

    I am always open to suggestions of new features. Just send me an email.

  16. I want to use one of your questionnaires. Can I get your permission to use it?

    I get this question quite a lot. Here are the rules:

    • Whenever you use PsyToolkit, you need to cite its two publications (see above)

    • Nobody is allowed to use PsyToolkit for commercial purposes or as part of a business, unless explicit permission has been given by me (this is rare).

    • Students can use PsyToolkit for educational or research purposes without further permission

    • The surveys are almost all based on work by others. I have checked that they can be used without violating copyright. In every online survey, there is a section about legal stuff. That is to the best of my knowledge. If you don’t trust it, please check yourself with the original study and its authors.

  17. How can I delete my account?

    This is very simple. There are 2 completely different ways to do this:

    1. Just do not login for one year. If you do not login for more than one year, you will be reminded by email to login. If you do still not login, your account will be deleted.

    2. Go to settings, and select "delete your account". You then receive an email with a code you need to enter here to finish the process.

About online experiments

  1. After running my experiment, I can see the data, but how can I save it to my harddisk?

    It depends what exactly you want. There are different scenario:

    1. When you just want to program and test your experiment, you cannot directly "save" your data to harddisk. You need to copy and paste it. This is due to security mechanism of the browser.

    2. If you want to run an online experiment as part of online data collection, you can embed your experiment in an online survey. That way, the data will be saved on the PsyToolkit server, and you can download them from there. Check the following question.

  2. I have my experiment. But how do I get my experiment "online" so that people can "click" it?

    This is very simple. It takes a few steps:

    • First make sure your experiment on its own works well.

    • In addition to your experiment, you also must create an online questionnaire.

    • The online questionnaire needs to embed your experiment, that is very simple to set up (see example below).

    • Now compile your survey and then you get a URL which you can distribute to people.

    • Imagine you have an experiment called "stroop". Then you can embed in your survey as follows:

      l: myexperiment
      t: experiment
      - stroop
      You can read about this in detail about how to get your experiment online here.
  3. Is the scripting language for online experiments different from the Linux version?

    No, it is the same, although some features are designed only for the Linux version (such as external keyboards) and some features only for online experiments

  4. How can I collect data from multiple participants?

    This is very easy. You need to run the experiment within an online survey. You need to set up an online survey and embed your experiment in it. Then you get an online web link (URL) and you can give that to your participants. All the data will be collected online and can be analyzed within the PsyToolkit website. A complete lesson on how to do this can be found here.

Online questionnaires

  1. How long can surveys run?

    Currently, they can be active for as long as you want, but there are no guarantees.

  2. Can you support my language?

    Currently, PsyToolkit supports many foreign languages. Contact me if it does not support your language. It is relatively easy to add new languages, as long as someone is willing to translate the basic phrases used.

  3. Will surveys stay valid if the software is updated?

    Yes, the old versions will stay active. If you compile a survey with a specific PsyToolkit-for-the-web version, this will remain active, even though the latest version of PsyToolkit might have new or changed features. This makes the life of researchers easier.

  4. Which browsers are supported?

    All modern browsers, including Internet Explorer version 10 and higher. Embedding experiments with keyboard input requires a real keyboard (touchscreens won’t work very well or not at all).

  5. Is there a welcome screen for surveys?

    If you create a survey, you can put welcome-information in the various text boxes under survey intro screen.

  6. Where can I add information for participants (such as plain language statement, etc)?

    If you create a survey, you can put this information in the various text boxes under survey intro screen.

  7. How do I put my university logo on the welcome screen?

    Here are the two steps how to do this:

    1. Find the weblink to your logo, such as http://my_university.edu/logo.jpg

    2. In the "About this survey" section, enter <img src='http://my_university.edu/logo.jpg'>

    3. Save and compile and it should show up

  8. What sort of survey items are available?

    There are many different items available, including Likert scales, choice questions, text boxes, embedded experiments and videos, etc. Check the online documentation under Question Types. You can also copy and paste existing scales from the survey library.

  9. Can the order of questions be randomized?

    Yes (read how to)

  10. Can you randomly assign participants to questionnaires?

    Yes (read how to)

  11. Is there a tracker of how far the participant has come in a survey?

    At the top of the web page, the percentage completed is indicated. This might not always be accurate if you include conditional elements (i.e., you cannot predict how long the survey will last if certain questions might be skipped depending on how some questions are being answered).

  12. I have a "check" question, but I would like that people can enter some text if they want to choose something that is not on my list. Can I do that, and if so, how?

    Yes, you can do that, click here to see how.

  13. Can people go back with the browser back button?

    Yes

  14. How can you embed an experiment in a survey?

    You will first need to have a working experiment, and then you can embed it in the "experiment" item. An example can be viewed here.

  15. How can you embed a video in a survey?

    You can embed a YouTube video. Here is an example.

  16. What types of cognitive tasks are already available?

    There are a number of cognitive tasks in the online library. You will still need to understand data analysis techniques to extract the data you want. If there are special tasks you are interested in, you can program them yourselves, or make a suggestion to me to implement it.

  17. How can I invited participants to participate in the survey I have created?

    When you create a survey and select it to go online, a URL link is provided. You need to send this to your participants yourselves or advertise somewhere on a website, the PsyToolkit website will not do that for you.

  18. What sort of format will the data be stored in?

    The data will be available in a raw format (text file) and in the common spreadsheet format CSV, which can be opened with all spreadsheet software. Excel export is also supported.

  19. Can surveys be programmed so that participants get feedback about their own answers at the end?

    Yes (using the set question, click here to read about it).

  20. Can the text font of questions be changed?

    Yes, all text can be changed using html tags. Click here for some examples.

  21. Is there an option for "must fill out" questions?

    Yes, you can "require" that people give an answer. For "radio" (choice) questions this is the default, and for text boxes you can add the option require. Check the documentation about textlines on how this works.

  22. Can you have survey links that are unique to participants?

    Each survey you create has one survey link. This means, in essence, that participants are unidentifyable. You can, of course, ask people to identify them with a question about their email. Also, you can give people an identifier code at the end of the survey, which they can then use for course credits, etc. Check here how that works.

  23. Can you run surveys in different languages?

    Yes, the text on the buttons and so on can be set to quite many different languages.

  24. Can you change the button text?

    Yes, you can use the questionnaire item operator "b:", see Check here how that works.

  25. Can I download my data in SPSS format?

    Under download, you can specify that you want to add Excel files, which can be easily and directly read into SPSS.

  26. I have a survey with an embedded experiment. How do I download the mean response times and error rates of participants?

    You need to make sure that you specify the parameters of the embedded experiment and then download again. You can do this after data collection. Click the blue questionmark button at "Download participant data" for detailed instructions.

Data storage

  1. Where are online data stored?

    Data are stored on the PsyToolkit webserver. This server is based in a Central European professional data center (Strasbourg).

  2. Can stored data be lost?

    This is very unlikely, because data are being backed up daily. If you delete a file on the server though, the backup will also delete it within 24 hours (so if you want it back, please contact me as soon as possible). PsyToolkit does not give any guarantees, though (see next point about what you can do to always be on the safe side).

  3. Can I store the data myself?

    Yes, you can download all your surveys, experiments, and data via the web interface. If you are concerned, just download your data yourself. As said, it is unlikely to get lost, but if your data is important to you, I recommend you to make sure that you make regular downloads. It is easy to do.

  4. Who has access to the data collected in online surveys?

    Only PsyToolkit developer Professor Dr. Gijsbert Stoet has access to all data, because he manages the PsyToolkit server.

  5. Is there any "no data loss" guarantee?

    Although data loss is extremely unlikely because of the distributed backup system (see above), you are responsible for your data. There are no guarantees given by PsyToolkit. Please note that you can download all collected data anytime to make a backup copy on your own computer at home. This is recommended.

  6. Are online survey participants informed about data storage?

    Online survey participants are informed about data storage at the bottom of the welcome survey of each online survey.

  7. Is the IP address or country information of participants stored?

    I sometimes get asked this because ethics review boards migth not like it if you store the IP address of a participant. Therefore, PsyToolkit gives you the option to save or not save the IP address of the participant. Please note the following as well: Some people overestimate the capacity to identify someone using an IP. In reality, IPs are not as useful as some people think they are. Even though IP addresses are linked to individual computers, it is typically practically impossible to identify who uses an IP. In theory, there will be law enforcement agencies who can, with time-consuming detective work, find out where a participant would be located, but the reality is that in practical terms, the IP address is still pretty anonymous and of little use except for finding out in which country the participant is. You can read more about this here.

Writing experiments

  1. What is the output data format of experiments?

    The output data format is ascii text. Further, it is entirely up to the designer of the script which variables are being save (with the save statement).

  2. Can PsyToolkit play a sound stimulus, like a short beep?

    Yes it can. See the statment sound and link

  3. Can PsyToolkit play some music when a participant is doing a block of trials?

    Yes it can do that too. See the statment sound.

  4. Can I include files into my script?

    Yes, see the include statement.

  5. Can PsyToolkit use functions?

    There are no "functions" which can pass parameters, but you can use the "part" syntax to use the same code snippets at multiple pages. Read here about the part statement.

Linux version

  1. Is there a recommended Linux distribution?

    No, PsyToolkit for Linux should run on any modern distribution. Mint Linux seems currently one of the most popular. Linux is tested mostly on Arch Linux, Mint, and Fedora.

  2. Where can I download the PsyToolkit software package for Linux?

    See http://www.psytoolkit.org/download for RPM, Debian, and source packages. It is very easy to install.

  3. Will you implement this on other platforms, such as Mac or Windows?

    No, but there is other free software available for those platforms.

  4. Which external keyboards does PsyToolkit support?

    Cedrus, BlackBox and IOlab (although these latter ones are probably no longer being made or sold). You can also build your own keyboard and use the parallel port. Read more about this here.

  5. Will PsyToolkit work with the new Wayland graphics stack?

    Yes. PsyToolkit uses the SDL (version 2) libraries for graphics and sound. SDL supports Wayland.