Tag: Research

20 Aug

How To Build An Ethical User Research Practice At Any Organization

by TBSCategories: News

Ethics are an important part of human-computer interaction because they keep people at the heart of the design process. Unethical practices at companies lead to the creation of technology that we all use but often neglect the influence of those intended to serve. As seen in the discussions about privacy at Facebook and AI-powered imaging at Google among other examples, when there are no ethical considerations during the creation of technology then the interaction between humans is compromised.

A real interaction between computers and humans starts at technological conception by understanding users through a sound and ethical user research discipline.

What Are “Ethics” And Why Should I Care?

To understand why we should build an ethical user research practice at our organizations, we’ll start with the basics. What are ethics anyway?

Ethics are the moral standards by which a person (or in this context, a company or team) should govern the behavior or the conduct of an activity. Ethical user research should then be about protecting the dignity, rights, and welfare of the participants from whom we are asking so much information or data. If we aren’t proactive and prescriptive about our own standards as a research team, we could unconsciously be biased, demoralizing, exclusive, or even in breach of the law.

Here are a few examples of unethical practices:

  • Not recruiting a diverse and inclusive sample.
  • Not informing participants of study requirements or potential harm.
  • Misplacing or not tracking participant data or identifiable information.
  • Misgendering or reinforcing stereotypes in reports, personas, or prototypes.

You may be wondering why you specifically should care. Doesn’t somebody else in the organization focus on ethics? Isn’t there an ethics Internal Review Board (IRB) that is better suited for this? Unfortunately in the user experience industry, that answer is usually no.

While many companies rely on their own Ethics or HR departments to set governing moral standards, their scope typically only relates to their business’ product offerings, services to customers, internal processes, and functions. Rarely, if ever, do these departments take into account the specialty of user research.

For example, try and recall the last time your team’s processes were audited for compliance against the company’s ethical standards. For this reason, it’s really up to each of us; it’s up to each and every practitioner to set standards to which ourselves, our teams, and our company should be held accountable.

Victor Yocco, an avid author and speaker on the intersection of psychology and research, provides a compelling case to self educate, attend training and sustain close review of our practice in his thoughtful article on “Ethical Considerations In UX Research: The Need For Training And Review”:

“Researchers by training and trade have often been required to take courses and pass exams to reflect an awareness of potential ethical issues in research. We can best prepare our colleagues to avoid these situations through similar training and standards.”

— Victor Yocco

Unfortunately, many organizations may not have the resources to fund such training, nor provide the time needed to invest in it. Even if your organization did have the time and resources to provide such training, then what? How do you get started leveraging ethics in your user research practice to influence how you do your work in a way that upholds a set of guiding standards? It all starts with a Code of Ethics.

“I do not believe in the immortality of the individual, and I consider ethics to be an exclusively human concern with no superhuman authority behind it.”

— Albert Einstein

A Code Of Ethics: The Backbone Of Ethical Practice

Now that you understand why ethics matter for user research, and why it’s critical that you take it upon yourself to create them for your team, it would certainly help knowing where to start. To begin influencing your work with a set of ethical standards, you need to start by defining what your standards even are.

A Code of Ethics is the standard by which you and others on the team who practice user research will be held accountable.

Starting with a Code versus diving headfirst into changing templates, tools, and processes that allow you to start from a reference point. A Code affords you with a checklist to ensure that with each new process, each new tool or each new study conducted follows your agreed-upon moral standards.

Once you create your own code of ethics, you might notice an improvement not only in your team’s work and performance but also in the team’s culture as well. As UX practitioners, we know empathy is an important part of the design process. Providing a deeper connection and purpose in the work we do and those who we do it with fosters a more inclusive environment, and thoughtful team members as a result.

How To Create Your Own Code Of Ethics

Creating a code of ethics takes only a few thoughtful steps, and from there you will have the backbone for influencing your work through a set of aligned standards. If you’re ready to invest in building a better, more ethical future for your team, your practice, and your organization, read on.

1. Commit To Ethics As A Practice

While it may sound silly or downright obvious, this is one of the most important and often most difficult of steps to truly accomplish. The thing is, to do things correctly aligned to any set of standards means being more thoughtful and taking the necessary steps and time to get it right.

While not monumental, ethical practice is an investment. It takes effort and time to evaluate your process, templates, tools, vendors, and ways of working in order to tweak or even overhaul them in order to align with your ethical principles or moral standards.

That said, building an ethical research practice makes good business sense, which means the time and effort is well spent when you purposely recruit diverse voices, respect their rights to information security and impartially represent them within your company’s walls.

2. Explore Existing Academic And Industry Examples

Why reinvent the wheel when we already have so many inspiring starting points? Once you’re committed to ethics, you should explore other relevant examples of research ethics in the wild. Many organizations proudly share their own code of ethics with the public, serving as a worthy starting point for your own.

It would benefit any team conducting this desk research to include both academic as well as industry examples. Academia tends to include IRBs, which I find are rarely practical for industry — it’s up to us to uphold the ethics of our practice, as previously mentioned.

Here are some great examples to get you started:

Take a ‘researcher approach’ to reviewing these examples by keeping a log of each organization’s ethical principles or guidelines. Put them each into a long word document or spreadsheet and collate across 15-20 examples. You’ll definitely notice a few strong patterns and a lot of overlap. These overlaps and key themes will serve as a jumping-off point for developing your own principles, relevant to your team, organization, and industry.

3. Familiarize Yourself With Relevant Laws

Whether only relevant to your state, union, or industry, you’ll want to make sure you’re intimately familiar with relevant laws and regulations that force compliance around a certain level of ethical practice.

For example, the General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA) both have strict rules surrounding how organizations may collect, store and secure consumer data. This not only applies to how companies do business but how they conduct research as well! If you live in the European Union or the state of California and are not yet familiar with these laws, I highly recommend you start reading today.

These regulations are actually quite profound as they almost codify ethics into law, requiring organizations to be compliant or otherwise face penalties such as fines or legal action. While these laws are based on ethical practice, they are still bare-bones and don’t cover the full spectrum of participant welfare or dignity, which are paramount when consumers are in your charge during the course of your research.

Some industries have their own regulations or legal practices which safeguard consumer data even more tightly than GDPR or CCPA. You may be familiar with the Health Insurance Portability and Accountability Act (HIPPA), which protects sensitive patient health information from being disclosed without consent or knowledge.

Many may not be familiar with other specific protections such as Customer Network Proprietary Information (CPNI) which safeguards the type, quantity, configuration, or location of telecommunications and interconnected services you use. Being mindful and educated about your industry’s specific regulations will not only keep you out of trouble but will provide yet another input into creating your own ethical user research principles.

4. Brainstorm Your Core Principles As A Team

Now the fun part! Whether your team is distributed or co-located, you’ll want to use what you’ve learned in steps 2 and 3 in order to brainstorm what principles you want to codify into your own code of ethics. There truly is no right or wrong way to do this, however, following a typical ideation workshop framework will really help add rigor to the process.

Start by affinity mapping your industry and academic examples using sticky notes, google slides or miro/mural board. Working as a team, identify the overlaps across multiple organizations by grouping the principles you feel are similar or otherwise saying the same thing. If you feel two principles are intertwined or tightly related, however, still distinct then by all means include both in the same principle.

For example “Diversity” or “Inclusion” have distinct goals, however, they’re often grouped together as “Diversity, Equity And Inclusion.” If there is strong disagreement between one or more principles that some feel should be combined while others feel should be separate, then run a dot-voting prioritization exercise. Each team member gets a dot and is allowed to cast their vote to either group or keep separate.

Then incorporate industry, state, or special laws/regulations from your research in step 3. Start by identifying any overlaps with your existing principles. If there’s overlap, simply build onto your existing principle! If you find a piece of the law is unique or demands its own dedicated principle, then by all means codify it that way!

Lastly, take a step back and look at your ethical principles as a whole set. Ideally, you would have no more than 10, no less than 4 principles — more than 10 and they’ll be hard to keep track of; less than 4 and creating specific and relevant examples when creating your Ethical Code will be difficult.

The goal is to have specific enough principles where everyone on the team can think of 2-3 examples of exactly how to apply that principle to their everyday work, without being overly general or vague. If you feel you have too many, consider if there are any overlaps or relationships between two or more and combine them accordingly into a larger principle.

Here’re example principles after going through the exercises above:

  • Sensitivity And No Harm,
  • Honesty And Transparency,
  • Confidentiality And Data,
  • Accuracy And Impartiality,
  • Diversity, Equity And Inclusion.

Once you’ve determined your high-level principles and feel good about them as a team, it’s time to make it real and draft your Code of Ethics.

5. Draft Your Code For Feedback

In order to make your Code of Ethics more easily understood, actionable and impactful, you’ll need to expand on each principle with three key components:

  1. A brief description
    This should expound upon the principle itself and provide more detail in its definition and intention. What does honesty mean in the content of user research? When defining this principle, it’s important to note why it’s relevant in the first place and why it is a moral standard. Within this description, you’ll also want to take a moment to identify who the principle is intended for. A team that follows an honorable Code of Ethics is no doubt benefiting its research participants, but it may also apply to internal processes and stakeholder teams as well.
  2. Bulleted agreements
    Perhaps the most important piece of the Code is where it would benefit you to get specific and directly relate to the user research practice at your organization. These agreements should not only exemplify the principle itself but provide more clarity as to how the team may apply the principle in their everyday work. In a way, these become somewhat of a checklist of tactics you’ll execute every time you plan, recruit for, conduct, and report on research.
  3. A few examples
    Like that which is provided below, examples help paint a full picture of the principle in action. You want to be careful to frame your example in the context of an ethical problem, then showcase the appropriate solution which clearly ties to an agreement of the principle. Even better if you have real past experiences that you can use as examples!

Putting these components of the Code together is a great place to have a roundtable discussion about the definition of each principle as well as individual researcher’s past experiences in unique ethical situations. It becomes extraordinarily fruitful to then have an open conversation around what the team believes should be the most appropriate definition, agreements, and examples. This process not only builds collective buy-in for the Code but also deepens everyone’s understanding of each principle as well.

Here’s an example principle in action:

  • Honesty And Transparency
    Maintains the integrity of our individual researchers and our research work. This principle applied not only to participants but also internally when discussing study design or findings.
    • Inform participants of study requirements upfront;
    • Inform participants where their data is going and how it will generally be used;
    • Explain any purposeful manipulation at a session’s end;
    • Explain the pros and cons of methodologies, and study limitations to internal stakeholders.

Example:
When designing a study with biometrics, devices with sensors need to be worn by participants. It should be communicated to participants that they will be wearing devices on or close to the skin which may cause minor irritation. The researcher should clearly explain why and where they’ll be needed in the study.

6. Implement Feedback And Finalize

When you have your first pass at your Code of Ethics and it shares collective buy-in from members of the team, it’s time to get some external validation from partner teams before considering it final.

In this step, you should schedule meetings with your points of contact from the legal and ethics departments within your organization in order to share the great work you’ve done as well as to check to make sure you’re not missing anything. It’s important to make sure you have this second pair of eyes because these groups are responsible for upholding ethics across the business and have deep knowledge of CCPA, GDPR, and other regulations which may affect your final Code.

Be warned that if you don’t already work closely with these teams, you may need to first introduce them to the type of work you do and the purpose of your team. This could mean two separate meetings; an introduction meeting may be needed to explain your team and a follow-on meeting to deep dive into your Code of Ethics.

If they’re unfamiliar with or not aware of any user research currently being done, they may already have strong concerns regarding privacy law. Remind them that’s why you’re creating this Code and why you’ve reached out to them — to ensure everyone is on the same page!

Putting It Into Practice

Now that you have a strong Code of Ethics that has internal and external team support, you’ll want to put it to work. The following are examples of where your Code of Ethics directly affects the processes, tools, and practices of your team. This is the part where the investment really comes in as each of these could take hours or even days to set the foundation for.

If you don’t proceed beyond this point and stop only when you’ve made your Code, then you run the risk of never really practicing what you’ve created. While the following isn’t quite ‘set it and forget it,’ rest assured the majority of the work is upfront in building the documentation vs. the ongoing maintenance and practice. Keep in mind these are great starting points, however, you may have other areas which could be improved using your new principles!

Screeners And Recruitment

Assuming you have an ethical principle-centered around or associated with Diversity, Equity, and Inclusion (DE&I), updating your screener questionnaire template and recruitment practices will help ensure every study accounts for the diversity and uniqueness of the customers for whom you serve.

In screener questionnaires, care should be taken in the administration and wording of questions relating to race, ethnicity, and gender. Because people identify in different and unique ways, these questions are asked and what responses you allow should reflect that flexibility. While there is plenty of literature out there to help you with crafting most demographics related questions, here are some ethical considerations:

  • Before asking for personal information such as gender or race, provide a very brief explanation as to why you’re asking for it in the first place. Providing this context can ease the minds of those who fear their information may be used for exploitation.
  • Allow ‘select all that apply’ checkbox answer options. Many may identify as mixed-race or on a spectrum of gender — providing them with multiple ways of identification more accurately represents them.
  • Provide a ‘prefer not to answer’ option for those who don’t feel comfortable disclosing this information.
  • For race, be exhaustive with the answer options to account for diverse classification and to avoid misrepresentation.
  • For gender, include ‘non-binary’ and ‘prefer to self identify’ options to accommodate those who don’t identify within a fixed gender.

In addition to screener wording, the individuals or team responsible for study recruitment efforts should make a purposeful effort to recruit a diverse group, even for small sizes for qualitative research. When working with recruitment firms or advanced panel tools, this should be relatively straightforward.

However, committing to diversity may mean canceling a few sessions and re-recruiting if your first round of recruitment efforts result in 9/10 participants being middle-aged white men, for example.

Data Retention And Management

Participants often provide sensitive, personally identifiable information that they may expect will be kept private and secure. This may even be a requirement of the law, as per the aforementioned regulations (GDPR And CCPA). In order to do this, you will need to create a comprehensive Data Retention Policy for all types of information that you’re collecting from participants. While there are many GDPR And CCPA templates and resources available to help you create your own policy, the key questions that must be asked are:

  • What data are you collecting?
  • Are you storing it? Why/for what purpose?
  • How long are you storing it for before deletion?
  • How is this data being secured?
  • Who, if anyone else, has access to this data?

Once you create your own Data And Retention policy, be sure to gain alignment with your own Legal team to ensure it meets the standards of local or federal requirements — especially in regards to specialized data related to CPNI or HIPPA. Then, share and train your team to uphold this policy, explaining the importance of safeguarding sensitive participant information.

An easy but effective way to hold everyone accountable to this policy is to include a ‘Data Retention Plan’ section within every Test Plan document which outlines what data is being collected during the course of the research, how it will be used, if it will be stored, and how it is being protected. For example, an in-situ contextual inquiry where COVID-19 vaccination status is being discussed, you’d want to document:

Data Instrument Retention Plan Notes
Name And Likeness Video/Audio Recording And Recruitment firm Codify as ID # Identification is not important. Participant will be anonymized.
Mailing Address Recruitment Firm Delete after data collection Only needed for moderator’s arrival during the study.
Vaccination Status Video/Audio Recording Maintain for 6 months Needed to create personas. Data not needed after study completion.

Informed Consent And NDAs

A key component of ethical user research is keeping the participant comfortable and informed at all times in order to protect their dignity and welfare. This may be done by providing them proper documentation and a forum in which to address questions or concerns that they may have prior to the study.

In addition, if a body of work is internally safeguarded or confidential, you may wish that the participants remain undisclosed. Non-disclosure Agreements must not be lengthy legal documents that are difficult to follow and require an advanced degree to read and understand.

There is more than one way to write an Informed Consent document, however, the key components are mostly the same:

  • Begin by thanking the participant for their consideration of participating in the study.
  • Explain the topic of the study at a high level by providing just enough detail to build an understanding of the subject matter.
  • Provide a clear, bulleted list of expectations or activities the participant will be required to participate in during their session.
  • Disclose any potential risk of harm, danger, or manipulation as part of the study.
  • Remind them of the time commitment for the session, any follow on activities after the study, and the nature of the incentive for their involvement.
  • End with a clear request to provide written consent of the above, including both their signature as well as today’s date.

It is paramount that participants are able to deny or withdraw their consent at any time, without warning or reason and without penalty — informed consent is useless if the participant is penalized or coerced into providing and maintaining their consent.

At times, our research may touch on triggering, emotional topics. Other times, there may be sudden emergencies that come up in the middle of the session. When these things happen, the participant should feel empowered to take a break or step away without fear of losing their incentive or fear of any form of retaliation.

Internal Representation: Reports, Prototypes And Personas

Ethical user research extends beyond the study and data collection, permeating into our internal artifacts that may be used for many months afterward. While there are more examples than will be covered here, the primary three areas where ethical considerations should be top of mind are in your findings reports, design prototypes, and especially in your user personas.

Findings Reports

When reporting a study’s findings, care should be taken to be as representative and impartial as you were during the design and administration of the study itself so that the audience or readers of the report have the full, accurate picture of what was learned and how.

  • Provide a ‘study setup’ section of the report which outlines a brief, but robust study methodology description. Include both Pros and Cons of the study method so that you’re being transparent in its limitations and reach.
  • Include a ‘participants’ section of the report which includes a bulleted list of the participant attributes. If necessary, include any footnotes around a limitation of diversity (i.e. if against all efforts you recruited 30 women of 40 total participants, this should be noted).
  • When using photos, avoid participant’s faces (likeness, PII) unless you’ve been provided explicit consent to do so from the participant. This is especially important when discussing personal or sensitive topics.
  • Use direct quotes, audio recordings, or photos from a range of participants to reflect and promote the diversity of your recruit. This provides the fullest picture of who you spoke with during the study.
  • Don’t paraphrase direct quotes nor infer too heavily without solid context or understanding. This maintains impartiality and avoids unconscious bias.
  • Avoid reporting demographic information for individual qualitative findings as it may inadvertently support institutionalized biases or stigma (i.e. 2 male Caucasians said X, 1 black woman said Y). It is best practice to segment by demographics only with quantitative samples where meaningful differences may be found through statistical analysis.

User Personas

A common framework for communicating generative user research findings is the user persona — an archetypical representation of a group of users that exhibit similar attitudes and behaviors. While incredibly powerful for making design decisions for months or even a year after the completion of research, they’re just as powerful in reinforcing gender, age, or racial stereotypes.

Care should be taken in the crafting of user personas so that they uphold their humanizing character while avoiding enforcing bias. While there are other resources for creating agnostic user persons, here are a few highlights:

  • Try to avoid using human names altogether. Using human names often promote biased stereotypes given strong societal expectations around different gender roles, positions of authority, or occupational status. Instead, you may lean into pithy 2-3 word titles which better exemplify who your Personas are (i.e. ‘Cautious Comparer,’ or ‘Impulsive Spender’).
  • Use gender-neutral and culturally diverse names. If you insist on using names in order to humanize the personas, use gender-neutral names to fight the unconscious gender bias. There are also many cultures and backgrounds with which your users may identify — using only names from western European etymology may incidentally erase their unique identity. Using names from diverse etymology helps fight this bias. Here are a few examples of gender-neutral, culturally diverse names: Adrian, Armani, Brett, Devon, Kai, Krishna, Maren, Noor, Nilam, Sam, Jaylin, Jordan, Yoshi.
  • As with the above, leverage artistic visualizations which exemplify the Persona more accurately than a fictitious image or photo. For example, in a Persona for a car manufacturer, it is much more telling to see a weighted scale with a dollar sign on one side and the word ‘safety’ on the other side than it is to see a stock photo of a worried looking person in the front seat. Using photos or illustrations for Personas further stigmatizes race, gender, and body type. If insistent on using humanizing imagery, consider artful illustrations which are ambiguous in gender, race, and body types in order to remain inclusive.
  • Do not include disability status unless this was directly part of your research or you’re creating a set of personas specifically focused on people of varying abilities or with a focus on intersectionality. Consider if this level of information or detail is important for the product you’re creating. While the goal of including this information may be with good intentions, you may be misrepresenting already marginalized communities.
  • Try and avoid gender, race, sexual orientation, or other demographics that are more akin to segmentation analysis and not user personas. Personas should be representations of like-attitudes and especially behaviors which often transcend basic demographics.

As apparent by the above guidance, creating Personas tows a very fine line between creating a useful fictitious archetype and promoting bias and stigma. The latter is highly unethical and leads to false assumptions and even worse design and content strategy decisions.

Design Prototypes

Similar to the above guidance given to User Personas, the same care should be taken with users personified through prototype designs. Often we create situations or scenarios to stress test the content and interaction design of our systems by placing a fictitious character at the heart of our mock-ups.

Consider leveraging the gender-neutral and culturally diverse names above for use within your prototypes (i.e. the account name of an authenticated user). For account or profile images, leverage artistic illustrations, or a diverse range of stock photography to depict your users. When creating designs where multiple individuals will be represented, take a step back to ensure diversity in the holistic group rather than a focus on one gender, race, culture, or body type.

Bringing It All Together

Ethical user research isn’t only the right thing to do but it makes good business sense. Principles of ethical conduct guide our decision-making, keeping us out of trouble and holding us accountable to our users and society.

By influencing your practices, templates, and processes with a foundation of ethical principles you will show your team and your organization that you value the dignity and welfare of your users. Building an ethical user research practice at any organization is not inherently difficult, however, requires upfront and ongoing conscious investment in order to do so effectively.

It’s up to us to put the humans we design for at the center of our own business practices and create a true form of human-computer interaction. No one will do this work for us. Noone will hold us accountable. After all, ethics (especially in our field) are an exclusively human concern with no superhuman authority behind it.

Are you ready to make a difference?

Articles on Smashing Magazine — For Web Designers And Developers

20 Aug

How To Build An Ethical User Research Practice At Any Organization

by TBSCategories: News

Ethics are an important part of human-computer interaction because they keep people at the heart of the design process. Unethical practices at companies lead to the creation of technology that we all use but often neglect the influence of those intended to serve. As seen in the discussions about privacy at Facebook and AI-powered imaging at Google among other examples, when there are no ethical considerations during the creation of technology then the interaction between humans is compromised.

A real interaction between computers and humans starts at technological conception by understanding users through a sound and ethical user research discipline.

What Are “Ethics” And Why Should I Care?

To understand why we should build an ethical user research practice at our organizations, we’ll start with the basics. What are ethics anyway?

Ethics are the moral standards by which a person (or in this context, a company or team) should govern the behavior or the conduct of an activity. Ethical user research should then be about protecting the dignity, rights, and welfare of the participants from whom we are asking so much information or data. If we aren’t proactive and prescriptive about our own standards as a research team, we could unconsciously be biased, demoralizing, exclusive, or even in breach of the law.

Here are a few examples of unethical practices:

  • Not recruiting a diverse and inclusive sample.
  • Not informing participants of study requirements or potential harm.
  • Misplacing or not tracking participant data or identifiable information.
  • Misgendering or reinforcing stereotypes in reports, personas, or prototypes.

You may be wondering why you specifically should care. Doesn’t somebody else in the organization focus on ethics? Isn’t there an ethics Internal Review Board (IRB) that is better suited for this? Unfortunately in the user experience industry, that answer is usually no.

While many companies rely on their own Ethics or HR departments to set governing moral standards, their scope typically only relates to their business’ product offerings, services to customers, internal processes, and functions. Rarely, if ever, do these departments take into account the specialty of user research.

For example, try and recall the last time your team’s processes were audited for compliance against the company’s ethical standards. For this reason, it’s really up to each of us; it’s up to each and every practitioner to set standards to which ourselves, our teams, and our company should be held accountable.

Victor Yocco, an avid author and speaker on the intersection of psychology and research, provides a compelling case to self educate, attend training and sustain close review of our practice in his thoughtful article on “Ethical Considerations In UX Research: The Need For Training And Review”:

“Researchers by training and trade have often been required to take courses and pass exams to reflect an awareness of potential ethical issues in research. We can best prepare our colleagues to avoid these situations through similar training and standards.”

— Victor Yocco

Unfortunately, many organizations may not have the resources to fund such training, nor provide the time needed to invest in it. Even if your organization did have the time and resources to provide such training, then what? How do you get started leveraging ethics in your user research practice to influence how you do your work in a way that upholds a set of guiding standards? It all starts with a Code of Ethics.

“I do not believe in the immortality of the individual, and I consider ethics to be an exclusively human concern with no superhuman authority behind it.”

— Albert Einstein

A Code Of Ethics: The Backbone Of Ethical Practice

Now that you understand why ethics matter for user research, and why it’s critical that you take it upon yourself to create them for your team, it would certainly help knowing where to start. To begin influencing your work with a set of ethical standards, you need to start by defining what your standards even are.

A Code of Ethics is the standard by which you and others on the team who practice user research will be held accountable.

Starting with a Code versus diving headfirst into changing templates, tools, and processes that allow you to start from a reference point. A Code affords you with a checklist to ensure that with each new process, each new tool or each new study conducted follows your agreed-upon moral standards.

Once you create your own code of ethics, you might notice an improvement not only in your team’s work and performance but also in the team’s culture as well. As UX practitioners, we know empathy is an important part of the design process. Providing a deeper connection and purpose in the work we do and those who we do it with fosters a more inclusive environment, and thoughtful team members as a result.

How To Create Your Own Code Of Ethics

Creating a code of ethics takes only a few thoughtful steps, and from there you will have the backbone for influencing your work through a set of aligned standards. If you’re ready to invest in building a better, more ethical future for your team, your practice, and your organization, read on.

1. Commit To Ethics As A Practice

While it may sound silly or downright obvious, this is one of the most important and often most difficult of steps to truly accomplish. The thing is, to do things correctly aligned to any set of standards means being more thoughtful and taking the necessary steps and time to get it right.

While not monumental, ethical practice is an investment. It takes effort and time to evaluate your process, templates, tools, vendors, and ways of working in order to tweak or even overhaul them in order to align with your ethical principles or moral standards.

That said, building an ethical research practice makes good business sense, which means the time and effort is well spent when you purposely recruit diverse voices, respect their rights to information security and impartially represent them within your company’s walls.

2. Explore Existing Academic And Industry Examples

Why reinvent the wheel when we already have so many inspiring starting points? Once you’re committed to ethics, you should explore other relevant examples of research ethics in the wild. Many organizations proudly share their own code of ethics with the public, serving as a worthy starting point for your own.

It would benefit any team conducting this desk research to include both academic as well as industry examples. Academia tends to include IRBs, which I find are rarely practical for industry — it’s up to us to uphold the ethics of our practice, as previously mentioned.

Here are some great examples to get you started:

Take a ‘researcher approach’ to reviewing these examples by keeping a log of each organization’s ethical principles or guidelines. Put them each into a long word document or spreadsheet and collate across 15-20 examples. You’ll definitely notice a few strong patterns and a lot of overlap. These overlaps and key themes will serve as a jumping-off point for developing your own principles, relevant to your team, organization, and industry.

3. Familiarize Yourself With Relevant Laws

Whether only relevant to your state, union, or industry, you’ll want to make sure you’re intimately familiar with relevant laws and regulations that force compliance around a certain level of ethical practice.

For example, the General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA) both have strict rules surrounding how organizations may collect, store and secure consumer data. This not only applies to how companies do business but how they conduct research as well! If you live in the European Union or the state of California and are not yet familiar with these laws, I highly recommend you start reading today.

These regulations are actually quite profound as they almost codify ethics into law, requiring organizations to be compliant or otherwise face penalties such as fines or legal action. While these laws are based on ethical practice, they are still bare-bones and don’t cover the full spectrum of participant welfare or dignity, which are paramount when consumers are in your charge during the course of your research.

Some industries have their own regulations or legal practices which safeguard consumer data even more tightly than GDPR or CCPA. You may be familiar with the Health Insurance Portability and Accountability Act (HIPPA), which protects sensitive patient health information from being disclosed without consent or knowledge.

Many may not be familiar with other specific protections such as Customer Network Proprietary Information (CPNI) which safeguards the type, quantity, configuration, or location of telecommunications and interconnected services you use. Being mindful and educated about your industry’s specific regulations will not only keep you out of trouble but will provide yet another input into creating your own ethical user research principles.

4. Brainstorm Your Core Principles As A Team

Now the fun part! Whether your team is distributed or co-located, you’ll want to use what you’ve learned in steps 2 and 3 in order to brainstorm what principles you want to codify into your own code of ethics. There truly is no right or wrong way to do this, however, following a typical ideation workshop framework will really help add rigor to the process.

Start by affinity mapping your industry and academic examples using sticky notes, google slides or miro/mural board. Working as a team, identify the overlaps across multiple organizations by grouping the principles you feel are similar or otherwise saying the same thing. If you feel two principles are intertwined or tightly related, however, still distinct then by all means include both in the same principle.

For example “Diversity” or “Inclusion” have distinct goals, however, they’re often grouped together as “Diversity, Equity And Inclusion.” If there is strong disagreement between one or more principles that some feel should be combined while others feel should be separate, then run a dot-voting prioritization exercise. Each team member gets a dot and is allowed to cast their vote to either group or keep separate.

Then incorporate industry, state, or special laws/regulations from your research in step 3. Start by identifying any overlaps with your existing principles. If there’s overlap, simply build onto your existing principle! If you find a piece of the law is unique or demands its own dedicated principle, then by all means codify it that way!

Lastly, take a step back and look at your ethical principles as a whole set. Ideally, you would have no more than 10, no less than 4 principles — more than 10 and they’ll be hard to keep track of; less than 4 and creating specific and relevant examples when creating your Ethical Code will be difficult.

The goal is to have specific enough principles where everyone on the team can think of 2-3 examples of exactly how to apply that principle to their everyday work, without being overly general or vague. If you feel you have too many, consider if there are any overlaps or relationships between two or more and combine them accordingly into a larger principle.

Here’re example principles after going through the exercises above:

  • Sensitivity And No Harm,
  • Honesty And Transparency,
  • Confidentiality And Data,
  • Accuracy And Impartiality,
  • Diversity, Equity And Inclusion.

Once you’ve determined your high-level principles and feel good about them as a team, it’s time to make it real and draft your Code of Ethics.

5. Draft Your Code For Feedback

In order to make your Code of Ethics more easily understood, actionable and impactful, you’ll need to expand on each principle with three key components:

  1. A brief description
    This should expound upon the principle itself and provide more detail in its definition and intention. What does honesty mean in the content of user research? When defining this principle, it’s important to note why it’s relevant in the first place and why it is a moral standard. Within this description, you’ll also want to take a moment to identify who the principle is intended for. A team that follows an honorable Code of Ethics is no doubt benefiting its research participants, but it may also apply to internal processes and stakeholder teams as well.
  2. Bulleted agreements
    Perhaps the most important piece of the Code is where it would benefit you to get specific and directly relate to the user research practice at your organization. These agreements should not only exemplify the principle itself but provide more clarity as to how the team may apply the principle in their everyday work. In a way, these become somewhat of a checklist of tactics you’ll execute every time you plan, recruit for, conduct, and report on research.
  3. A few examples
    Like that which is provided below, examples help paint a full picture of the principle in action. You want to be careful to frame your example in the context of an ethical problem, then showcase the appropriate solution which clearly ties to an agreement of the principle. Even better if you have real past experiences that you can use as examples!

Putting these components of the Code together is a great place to have a roundtable discussion about the definition of each principle as well as individual researcher’s past experiences in unique ethical situations. It becomes extraordinarily fruitful to then have an open conversation around what the team believes should be the most appropriate definition, agreements, and examples. This process not only builds collective buy-in for the Code but also deepens everyone’s understanding of each principle as well.

Here’s an example principle in action:

  • Honesty And Transparency
    Maintains the integrity of our individual researchers and our research work. This principle applied not only to participants but also internally when discussing study design or findings.
    • Inform participants of study requirements upfront;
    • Inform participants where their data is going and how it will generally be used;
    • Explain any purposeful manipulation at a session’s end;
    • Explain the pros and cons of methodologies, and study limitations to internal stakeholders.

Example:
When designing a study with biometrics, devices with sensors need to be worn by participants. It should be communicated to participants that they will be wearing devices on or close to the skin which may cause minor irritation. The researcher should clearly explain why and where they’ll be needed in the study.

6. Implement Feedback And Finalize

When you have your first pass at your Code of Ethics and it shares collective buy-in from members of the team, it’s time to get some external validation from partner teams before considering it final.

In this step, you should schedule meetings with your points of contact from the legal and ethics departments within your organization in order to share the great work you’ve done as well as to check to make sure you’re not missing anything. It’s important to make sure you have this second pair of eyes because these groups are responsible for upholding ethics across the business and have deep knowledge of CCPA, GDPR, and other regulations which may affect your final Code.

Be warned that if you don’t already work closely with these teams, you may need to first introduce them to the type of work you do and the purpose of your team. This could mean two separate meetings; an introduction meeting may be needed to explain your team and a follow-on meeting to deep dive into your Code of Ethics.

If they’re unfamiliar with or not aware of any user research currently being done, they may already have strong concerns regarding privacy law. Remind them that’s why you’re creating this Code and why you’ve reached out to them — to ensure everyone is on the same page!

Putting It Into Practice

Now that you have a strong Code of Ethics that has internal and external team support, you’ll want to put it to work. The following are examples of where your Code of Ethics directly affects the processes, tools, and practices of your team. This is the part where the investment really comes in as each of these could take hours or even days to set the foundation for.

If you don’t proceed beyond this point and stop only when you’ve made your Code, then you run the risk of never really practicing what you’ve created. While the following isn’t quite ‘set it and forget it,’ rest assured the majority of the work is upfront in building the documentation vs. the ongoing maintenance and practice. Keep in mind these are great starting points, however, you may have other areas which could be improved using your new principles!

Screeners And Recruitment

Assuming you have an ethical principle-centered around or associated with Diversity, Equity, and Inclusion (DE&I), updating your screener questionnaire template and recruitment practices will help ensure every study accounts for the diversity and uniqueness of the customers for whom you serve.

In screener questionnaires, care should be taken in the administration and wording of questions relating to race, ethnicity, and gender. Because people identify in different and unique ways, these questions are asked and what responses you allow should reflect that flexibility. While there is plenty of literature out there to help you with crafting most demographics related questions, here are some ethical considerations:

  • Before asking for personal information such as gender or race, provide a very brief explanation as to why you’re asking for it in the first place. Providing this context can ease the minds of those who fear their information may be used for exploitation.
  • Allow ‘select all that apply’ checkbox answer options. Many may identify as mixed-race or on a spectrum of gender — providing them with multiple ways of identification more accurately represents them.
  • Provide a ‘prefer not to answer’ option for those who don’t feel comfortable disclosing this information.
  • For race, be exhaustive with the answer options to account for diverse classification and to avoid misrepresentation.
  • For gender, include ‘non-binary’ and ‘prefer to self identify’ options to accommodate those who don’t identify within a fixed gender.

In addition to screener wording, the individuals or team responsible for study recruitment efforts should make a purposeful effort to recruit a diverse group, even for small sizes for qualitative research. When working with recruitment firms or advanced panel tools, this should be relatively straightforward.

However, committing to diversity may mean canceling a few sessions and re-recruiting if your first round of recruitment efforts result in 9/10 participants being middle-aged white men, for example.

Data Retention And Management

Participants often provide sensitive, personally identifiable information that they may expect will be kept private and secure. This may even be a requirement of the law, as per the aforementioned regulations (GDPR And CCPA). In order to do this, you will need to create a comprehensive Data Retention Policy for all types of information that you’re collecting from participants. While there are many GDPR And CCPA templates and resources available to help you create your own policy, the key questions that must be asked are:

  • What data are you collecting?
  • Are you storing it? Why/for what purpose?
  • How long are you storing it for before deletion?
  • How is this data being secured?
  • Who, if anyone else, has access to this data?

Once you create your own Data And Retention policy, be sure to gain alignment with your own Legal team to ensure it meets the standards of local or federal requirements — especially in regards to specialized data related to CPNI or HIPPA. Then, share and train your team to uphold this policy, explaining the importance of safeguarding sensitive participant information.

An easy but effective way to hold everyone accountable to this policy is to include a ‘Data Retention Plan’ section within every Test Plan document which outlines what data is being collected during the course of the research, how it will be used, if it will be stored, and how it is being protected. For example, an in-situ contextual inquiry where COVID-19 vaccination status is being discussed, you’d want to document:

Data Instrument Retention Plan Notes
Name And Likeness Video/Audio Recording And Recruitment firm Codify as ID # Identification is not important. Participant will be anonymized.
Mailing Address Recruitment Firm Delete after data collection Only needed for moderator’s arrival during the study.
Vaccination Status Video/Audio Recording Maintain for 6 months Needed to create personas. Data not needed after study completion.

Informed Consent And NDAs

A key component of ethical user research is keeping the participant comfortable and informed at all times in order to protect their dignity and welfare. This may be done by providing them proper documentation and a forum in which to address questions or concerns that they may have prior to the study.

In addition, if a body of work is internally safeguarded or confidential, you may wish that the participants remain undisclosed. Non-disclosure Agreements must not be lengthy legal documents that are difficult to follow and require an advanced degree to read and understand.

There is more than one way to write an Informed Consent document, however, the key components are mostly the same:

  • Begin by thanking the participant for their consideration of participating in the study.
  • Explain the topic of the study at a high level by providing just enough detail to build an understanding of the subject matter.
  • Provide a clear, bulleted list of expectations or activities the participant will be required to participate in during their session.
  • Disclose any potential risk of harm, danger, or manipulation as part of the study.
  • Remind them of the time commitment for the session, any follow on activities after the study, and the nature of the incentive for their involvement.
  • End with a clear request to provide written consent of the above, including both their signature as well as today’s date.

It is paramount that participants are able to deny or withdraw their consent at any time, without warning or reason and without penalty — informed consent is useless if the participant is penalized or coerced into providing and maintaining their consent.

At times, our research may touch on triggering, emotional topics. Other times, there may be sudden emergencies that come up in the middle of the session. When these things happen, the participant should feel empowered to take a break or step away without fear of losing their incentive or fear of any form of retaliation.

Internal Representation: Reports, Prototypes And Personas

Ethical user research extends beyond the study and data collection, permeating into our internal artifacts that may be used for many months afterward. While there are more examples than will be covered here, the primary three areas where ethical considerations should be top of mind are in your findings reports, design prototypes, and especially in your user personas.

Findings Reports

When reporting a study’s findings, care should be taken to be as representative and impartial as you were during the design and administration of the study itself so that the audience or readers of the report have the full, accurate picture of what was learned and how.

  • Provide a ‘study setup’ section of the report which outlines a brief, but robust study methodology description. Include both Pros and Cons of the study method so that you’re being transparent in its limitations and reach.
  • Include a ‘participants’ section of the report which includes a bulleted list of the participant attributes. If necessary, include any footnotes around a limitation of diversity (i.e. if against all efforts you recruited 30 women of 40 total participants, this should be noted).
  • When using photos, avoid participant’s faces (likeness, PII) unless you’ve been provided explicit consent to do so from the participant. This is especially important when discussing personal or sensitive topics.
  • Use direct quotes, audio recordings, or photos from a range of participants to reflect and promote the diversity of your recruit. This provides the fullest picture of who you spoke with during the study.
  • Don’t paraphrase direct quotes nor infer too heavily without solid context or understanding. This maintains impartiality and avoids unconscious bias.
  • Avoid reporting demographic information for individual qualitative findings as it may inadvertently support institutionalized biases or stigma (i.e. 2 male Caucasians said X, 1 black woman said Y). It is best practice to segment by demographics only with quantitative samples where meaningful differences may be found through statistical analysis.

User Personas

A common framework for communicating generative user research findings is the user persona — an archetypical representation of a group of users that exhibit similar attitudes and behaviors. While incredibly powerful for making design decisions for months or even a year after the completion of research, they’re just as powerful in reinforcing gender, age, or racial stereotypes.

Care should be taken in the crafting of user personas so that they uphold their humanizing character while avoiding enforcing bias. While there are other resources for creating agnostic user persons, here are a few highlights:

  • Try to avoid using human names altogether. Using human names often promote biased stereotypes given strong societal expectations around different gender roles, positions of authority, or occupational status. Instead, you may lean into pithy 2-3 word titles which better exemplify who your Personas are (i.e. ‘Cautious Comparer,’ or ‘Impulsive Spender’).
  • Use gender-neutral and culturally diverse names. If you insist on using names in order to humanize the personas, use gender-neutral names to fight the unconscious gender bias. There are also many cultures and backgrounds with which your users may identify — using only names from western European etymology may incidentally erase their unique identity. Using names from diverse etymology helps fight this bias. Here are a few examples of gender-neutral, culturally diverse names: Adrian, Armani, Brett, Devon, Kai, Krishna, Maren, Noor, Nilam, Sam, Jaylin, Jordan, Yoshi.
  • As with the above, leverage artistic visualizations which exemplify the Persona more accurately than a fictitious image or photo. For example, in a Persona for a car manufacturer, it is much more telling to see a weighted scale with a dollar sign on one side and the word ‘safety’ on the other side than it is to see a stock photo of a worried looking person in the front seat. Using photos or illustrations for Personas further stigmatizes race, gender, and body type. If insistent on using humanizing imagery, consider artful illustrations which are ambiguous in gender, race, and body types in order to remain inclusive.
  • Do not include disability status unless this was directly part of your research or you’re creating a set of personas specifically focused on people of varying abilities or with a focus on intersectionality. Consider if this level of information or detail is important for the product you’re creating. While the goal of including this information may be with good intentions, you may be misrepresenting already marginalized communities.
  • Try and avoid gender, race, sexual orientation, or other demographics that are more akin to segmentation analysis and not user personas. Personas should be representations of like-attitudes and especially behaviors which often transcend basic demographics.

As apparent by the above guidance, creating Personas tows a very fine line between creating a useful fictitious archetype and promoting bias and stigma. The latter is highly unethical and leads to false assumptions and even worse design and content strategy decisions.

Design Prototypes

Similar to the above guidance given to User Personas, the same care should be taken with users personified through prototype designs. Often we create situations or scenarios to stress test the content and interaction design of our systems by placing a fictitious character at the heart of our mock-ups.

Consider leveraging the gender-neutral and culturally diverse names above for use within your prototypes (i.e. the account name of an authenticated user). For account or profile images, leverage artistic illustrations, or a diverse range of stock photography to depict your users. When creating designs where multiple individuals will be represented, take a step back to ensure diversity in the holistic group rather than a focus on one gender, race, culture, or body type.

Bringing It All Together

Ethical user research isn’t only the right thing to do but it makes good business sense. Principles of ethical conduct guide our decision-making, keeping us out of trouble and holding us accountable to our users and society.

By influencing your practices, templates, and processes with a foundation of ethical principles you will show your team and your organization that you value the dignity and welfare of your users. Building an ethical user research practice at any organization is not inherently difficult, however, requires upfront and ongoing conscious investment in order to do so effectively.

It’s up to us to put the humans we design for at the center of our own business practices and create a true form of human-computer interaction. No one will do this work for us. Noone will hold us accountable. After all, ethics (especially in our field) are an exclusively human concern with no superhuman authority behind it.

Are you ready to make a difference?

Articles on Smashing Magazine — For Web Designers And Developers

19 Aug

How To Build An Ethical User Research Practice At Any Organization

by TBSCategories: News

Ethics are an important part of human-computer interaction because they keep people at the heart of the design process. Unethical practices at companies lead to the creation of technology that we all use but often neglect the influence of those intended to serve. As seen in the discussions about privacy at Facebook and AI-powered imaging at Google among other examples, when there are no ethical considerations during the creation of technology then the interaction between humans is compromised.

A real interaction between computers and humans starts at technological conception by understanding users through a sound and ethical user research discipline.

What Are “Ethics” And Why Should I Care?

To understand why we should build an ethical user research practice at our organizations, we’ll start with the basics. What are ethics anyway?

Ethics are the moral standards by which a person (or in this context, a company or team) should govern the behavior or the conduct of an activity. Ethical user research should then be about protecting the dignity, rights, and welfare of the participants from whom we are asking so much information or data. If we aren’t proactive and prescriptive about our own standards as a research team, we could unconsciously be biased, demoralizing, exclusive, or even in breach of the law.

Here are a few examples of unethical practices:

  • Not recruiting a diverse and inclusive sample.
  • Not informing participants of study requirements or potential harm.
  • Misplacing or not tracking participant data or identifiable information.
  • Misgendering or reinforcing stereotypes in reports, personas, or prototypes.

You may be wondering why you specifically should care. Doesn’t somebody else in the organization focus on ethics? Isn’t there an ethics Internal Review Board (IRB) that is better suited for this? Unfortunately in the user experience industry, that answer is usually no.

While many companies rely on their own Ethics or HR departments to set governing moral standards, their scope typically only relates to their business’ product offerings, services to customers, internal processes, and functions. Rarely, if ever, do these departments take into account the specialty of user research.

For example, try and recall the last time your team’s processes were audited for compliance against the company’s ethical standards. For this reason, it’s really up to each of us; it’s up to each and every practitioner to set standards to which ourselves, our teams, and our company should be held accountable.

Victor Yocco, an avid author and speaker on the intersection of psychology and research, provides a compelling case to self educate, attend training and sustain close review of our practice in his thoughtful article on “Ethical Considerations In UX Research: The Need For Training And Review”:

“Researchers by training and trade have often been required to take courses and pass exams to reflect an awareness of potential ethical issues in research. We can best prepare our colleagues to avoid these situations through similar training and standards.”

— Victor Yocco

Unfortunately, many organizations may not have the resources to fund such training, nor provide the time needed to invest in it. Even if your organization did have the time and resources to provide such training, then what? How do you get started leveraging ethics in your user research practice to influence how you do your work in a way that upholds a set of guiding standards? It all starts with a Code of Ethics.

“I do not believe in the immortality of the individual, and I consider ethics to be an exclusively human concern with no superhuman authority behind it.”

— Albert Einstein

A Code Of Ethics: The Backbone Of Ethical Practice

Now that you understand why ethics matter for user research, and why it’s critical that you take it upon yourself to create them for your team, it would certainly help knowing where to start. To begin influencing your work with a set of ethical standards, you need to start by defining what your standards even are.

A Code of Ethics is the standard by which you and others on the team who practice user research will be held accountable.

Starting with a Code versus diving headfirst into changing templates, tools, and processes that allow you to start from a reference point. A Code affords you with a checklist to ensure that with each new process, each new tool or each new study conducted follows your agreed-upon moral standards.

Once you create your own code of ethics, you might notice an improvement not only in your team’s work and performance but also in the team’s culture as well. As UX practitioners, we know empathy is an important part of the design process. Providing a deeper connection and purpose in the work we do and those who we do it with fosters a more inclusive environment, and thoughtful team members as a result.

How To Create Your Own Code Of Ethics

Creating a code of ethics takes only a few thoughtful steps, and from there you will have the backbone for influencing your work through a set of aligned standards. If you’re ready to invest in building a better, more ethical future for your team, your practice, and your organization, read on.

1. Commit To Ethics As A Practice

While it may sound silly or downright obvious, this is one of the most important and often most difficult of steps to truly accomplish. The thing is, to do things correctly aligned to any set of standards means being more thoughtful and taking the necessary steps and time to get it right.

While not monumental, ethical practice is an investment. It takes effort and time to evaluate your process, templates, tools, vendors, and ways of working in order to tweak or even overhaul them in order to align with your ethical principles or moral standards.

That said, building an ethical research practice makes good business sense, which means the time and effort is well spent when you purposely recruit diverse voices, respect their rights to information security and impartially represent them within your company’s walls.

2. Explore Existing Academic And Industry Examples

Why reinvent the wheel when we already have so many inspiring starting points? Once you’re committed to ethics, you should explore other relevant examples of research ethics in the wild. Many organizations proudly share their own code of ethics with the public, serving as a worthy starting point for your own.

It would benefit any team conducting this desk research to include both academic as well as industry examples. Academia tends to include IRBs, which I find are practical for industry — it’s up to us to uphold the ethics of our practice, as previously mentioned.

Here are some great examples to get you started:

Take a ‘researcher approach’ to reviewing these examples by keeping a log of each organization’s ethical principles or guidelines. Put them each into a long word document or spreadsheet and collate across 15-20 examples. You’ll definitely notice a few strong patterns and a lot of overlap. These overlaps and key themes will serve as a jumping-off point for developing your own principles, relevant to your team, organization, and industry.

3. Familiarize Yourself With Relevant Laws

Whether only relevant to your state, union, or industry, you’ll want to make sure you’re intimately familiar with relevant laws and regulations that force compliance around a certain level of ethical practice.

For example, the General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA) both have strict rules surrounding how organizations may collect, store and secure consumer data. This not only applies to how companies do business but how they conduct research as well! If you live in the European Union or the state of California and are not yet familiar with these laws, I highly recommend you start reading today.

These regulations are actually quite profound as they almost codify ethics into law, requiring organizations to be compliant or otherwise face penalties such as fines or legal action. While these laws are based on ethical practice, they are still bare-bones and don’t cover the full spectrum of participant welfare or dignity, which are paramount when consumers are in your charge during the course of your research.

Some industries have their own regulations or legal practices which safeguard consumer data even more tightly than GDPR or CCPA. You may be familiar with the Health Insurance Portability and Accountability Act (HIPPA), which protects sensitive patient health information from being disclosed without consent or knowledge.

Many may not be familiar with other specific protections such as Customer Network Proprietary Information (CPNI) which safeguards the type, quantity, configuration, or location of telecommunications and interconnected services you use. Being mindful and educated about your industry’s specific regulations will not only keep you out of trouble but will provide yet another input into creating your own ethical user research principles.

4. Brainstorm Your Core Principles As A Team

Now the fun part! Whether your team is distributed or co-located, you’ll want to use what you’ve learned in steps 2 and 3 in order to brainstorm what principles you want to codify into your own code of ethics. There truly is no right or wrong way to do this, however, following a typical ideation workshop framework will really help add rigor to the process.

Start by affinity mapping your industry and academic examples using sticky notes, google slides or miro/mural board. Working as a team, identify the overlaps across multiple organizations by grouping the principles you feel are similar or otherwise saying the same thing. If you feel two principles are intertwined or tightly related, however, still distinct then by all means include both in the same principle.

For example “Diversity” or “Inclusion” have distinct goals, however, they’re often grouped together as “Diversity, Equity And Inclusion.” If there is strong disagreement between one or more principles that some feel should be combined while others feel should be separate, then run a dot-voting prioritization exercise. Each team member gets a dot and is allowed to cast their vote to either group or keep separate.

Then incorporate industry, state, or special laws/regulations from your research in step 3. Start by identifying any overlaps with your existing principles. If there’s overlap, simply build onto your existing principle! If you find a piece of the law is unique or demands its own dedicated principle, then by all means codify it that way!

Lastly, take a step back and look at your ethical principles as a whole set. Ideally, you would have no more than 10, no less than 4 principles — more than 10 and they’ll be hard to keep track of; less than 4 and creating specific and relevant examples when creating your Ethical Code will be difficult.

The goal is to have specific enough principles where everyone on the team can think of 2-3 examples of exactly how to apply that principle to their everyday work, without being overly general or vague. If you feel you have too many, consider if there are any overlaps or relationships between two or more and combine them accordingly into a larger principle.

Here’re example principles after going through the exercises above:

  • Sensitivity And No Harm,
  • Honesty And Transparency,
  • Confidentiality And Data,
  • Accuracy And Impartiality,
  • Diversity, Equity And Inclusion.

Once you’ve determined your high-level principles and feel good about them as a team, it’s time to make it real and draft your Code of Ethics.

5. Draft Your Code For Feedback

In order to make your Code of Ethics more easily understood, actionable and impactful, you’ll need to expand on each principle with three key components:

  1. A brief description
    This should expound upon the principle itself and provide more detail in its definition and intention. What does honesty mean in the content of user research? When defining this principle, it’s important to note why it’s relevant in the first place and why it is a moral standard. Within this description, you’ll also want to take a moment to identify who the principle is intended for. A team that follows an honorable Code of Ethics is no doubt benefiting its research participants, but it may also apply to internal processes and stakeholder teams as well.
  2. Bulleted agreements
    Perhaps the most important piece of the Code is where it would benefit you to get specific and directly relate to the user research practice at your organization. These agreements should not only exemplify the principle itself but provide more clarity as to how the team may apply the principle in their everyday work. In a way, these become somewhat of a checklist of tactics you’ll execute every time you plan, recruit for, conduct, and report on research.
  3. A few examples
    Like that which is provided below, examples help paint a full picture of the principle in action. You want to be careful to frame your example in the context of an ethical problem, then showcase the appropriate solution which clearly ties to an agreement of the principle. Even better if you have real past experiences that you can use as examples!

Putting these components of the Code together is a great place to have a roundtable discussion about the definition of each principle as well as individual researcher’s past experiences in unique ethical situations. It becomes extraordinarily fruitful to then have an open conversation around what the team believes should be the most appropriate definition, agreements, and examples. This team not only builds collective buy-in for the Code but also deepens everyone’s understanding of each principle as well.

Here’s an example principle in action:

  • Honesty And Transparency
    Maintains the integrity of our individual researchers and our research work. This principle applied not only to participants but also internally when discussing study design or findings.
    • Inform participants of study requirements upfront;
    • Inform participants where their data is going and how it will generally be used;
    • Explain any purposeful manipulation at a session’s end;
    • Explain the pros and cons of methodologies, and study limitations to internal stakeholders.

Example:
When designing a study with biometrics, devices with sensors need to be worn by participants. It should be communicated to participants that they will be wearing devices on or close to the skin which may cause minor irritation. The researcher should clearly explain why and where they’ll be needed in the study.

6. Implement Feedback And Finalize

When you have your first pass at your Code of Ethics and it shares collective buy-in from members of the team, it’s time to get some external validation from partner teams before considering it final.

In this step, you should schedule meetings with your points of contact from the legal and ethics departments within your organization in order to share the great work you’ve done as well as to check to make sure you’re not missing anything. It’s important to make sure you have this second pair of eyes because these groups are responsible for upholding ethics across the business and have deep knowledge of CCPA, GDPR, and other regulations which may affect your final Code.

Be warned that if you don’t already work closely with these teams, you may need to first introduce them to the type of work you do and the purpose of your team. This could mean two separate meetings; an introduction meeting may be needed to explain your team and a follow-on meeting to deep dive into your Code of Ethics.

If they’re unfamiliar with or not aware of any user research currently being done, they may already have strong concerns regarding privacy law. Remind them that’s why you’re creating this Code and why you’ve reached out to them — to ensure everyone is on the same page!

Putting It Into Practice

Now that you have a strong Code of Ethics that has internal and external team support, you’ll want to put it to work. The following are examples of where your Code of Ethics directly affects the processes, tools, and practices of your team. This is the part where the investment really comes in as each of these could take hours or even days to set the foundation for.

If you don’t proceed beyond this point and stop only when you’ve made your Code, then you run the risk of never really practicing what you’ve created. While the following isn’t quite ‘set it and forget it,’ rest assured the majority of the work is upfront in building the documentation vs. the ongoing maintenance and practice. Keep in mind these are great starting points, however, you may have other areas which could be improved using your new principles!

Screeners And Recruitment

Assuming you have an ethical principle-centered around or associated with Diversity, Equity, and Inclusion (DE&I), updating your screener questionnaire template and recruitment practices will help ensure every study accounts for the diversity and uniqueness of the customers for whom you serve.

In screener questionnaires, care should be taken in the administration and wording of questions relating to race, ethnicity, and gender. Because people identify in different and unique ways, these questions are asked and what responses you allow should reflect that flexibility. While there is plenty of literature out there to help you with crafting most demographics related questions, here are some ethical considerations:

  • Before asking for personal information such as gender or race, provide a very brief explanation as to why you’re asking for it in the first place. Providing this context can ease the minds of those who fear their information may be used for exploitation.
  • Allow ‘select all that apply’ checkbox answer options. Many may identify as mixed-race or on a spectrum of gender — providing them with multiple ways of identification more accurately represents them.
  • Provide a ‘prefer not to answer’ option for those who don’t feel comfortable disclosing this information.
  • For race, be exhaustive with the answer options to account for diverse classification and to avoid misrepresentation.
  • For gender, include ‘non-binary’ and ‘prefer to self identify’ options to accommodate those who don’t identify within a fixed gender.

In addition to screener wording, the individuals or team responsible for study recruitment efforts should make a purposeful effort to recruit a diverse group, even for small sizes for qualitative research. When working with recruitment firms or advanced panel tools, this should be relatively straightforward.

However, committing to diversity may mean canceling a few sessions and re-recruiting if your first round of recruitment efforts result in 9/10 participants being middle-aged white men, for example.

Data Retention And Management

Participants often provide sensitive, personally identifiable information that they may expect will be kept private and secure. This may even be a requirement of the law, as per the aforementioned regulations (GDPR And CCPA). In order to do this, you will need to create a comprehensive Data Retention Policy for all types of information that you’re collecting from participants. While there are many GDPR And CCPA templates and resources available to help you create your own policy, the key questions that must be asked are:

  • What data are you collecting?
  • Are you storing it? Why/for what purpose?
  • How long are you storing it for before deletion?
  • How is this data being secured?
  • Who, if anyone else, has access to this data?

Once you create your own Data And Retention policy, be sure to gain alignment with your own Legal team to ensure it meets the standards of local or federal requirements — especially in regards to specialized data related to CPNI or HIPPA. Then, share and train your team to uphold this policy, explaining the importance of safeguarding sensitive participant information.

An easy but effective way to hold everyone accountable to this policy is to include a ‘Data Retention Plan’ section within every Test Plan document which outlines what data is being collected during the course of the research, how it will be used, if it will be stored, and how it is being protected. For example, an in-situ contextual inquiry where COVID-19 vaccination status is being discussed, you’d want to document:

Data Instrument Retention Plan Notes
Name And Likeness Video/Audio Recording And Recruitment firm Codify as ID # Identification is not important. Participant will be anonymized.
Mailing Address Recruitment Firm Delete after data collection Only needed for moderator’s arrival during the study.
Vaccination Status Video/Audio Recording Maintain for 6 months Needed to create personas. Data not needed after study completion.

Informed Consent And NDAs

A key component of ethical user research is keeping the participant comfortable and informed at all times in order to protect their dignity and welfare. This may be done by providing them proper documentation and a forum in which to address questions or concerns that they may have prior to the study.

In addition, if a body of work is internally safeguarded or confidential, you may wish that the participants remain undisclosed. Non-disclosure Agreements must not be lengthy legal documents that are difficult to follow and require an advanced degree to read and understand.

There is more than one way to write an Informed Consent document, however, the key components are mostly the same:

  • Begin by thanking the participant for their consideration of participating in the study.
  • Explain the topic of the study at a high level by providing just enough detail to build an understanding of the subject matter.
  • Provide a clear, bulleted list of expectations or activities the participant will be required to participate in during their session.
  • Disclose any potential risk of harm, danger, or manipulation as part of the study.
  • Remind them of the time commitment for the session, any follow on activities after the study, and the nature of the incentive for their involvement.
  • End with a clear request to provide written consent of the above, including both their signature as well as today’s date.

It is paramount that participants are able to deny or withdraw their consent at any time, without warning or reason and without penalty — informed consent is useless if the participant is penalized or coerced into providing and maintaining their consent.

At times, our research may touch on triggering, emotional topics. Other times, there may be sudden emergencies that come up in the middle of the session. When these things happen, the participant should feel empowered to take a break or step away without fear of losing their incentive or fear of any form of retaliation.

Internal Representation: Reports, Prototypes And Personas

Ethical user research extends beyond the study and data collection, permeating into our internal artifacts that may be used for many months afterward. While there are more examples than will be covered here, the primary three areas where ethical considerations should be top of mind are in your findings reports, design prototypes, and especially in your user personas.

Findings Reports

When reporting a study’s findings, care should be taken to be as representative and impartial as you were during the design and administration of the study itself so that the audience or readers of the report have the full, accurate picture of what was learned and how.

  • Provide a ‘study setup’ section of the report which outlines a brief, but robust study methodology description. Include both Pros and Cons of the study method so that you’re being transparent in its limitations and reach.
  • Include a ‘participants’ section of the report which includes a bulleted list of the participant attributes. If necessary, include any footnotes around a limitation of diversity (i.e. if against all efforts you recruited 30 women of 40 total participants, this should be noted).
  • When using photos, avoid participant’s faces (likeness, PII) unless you’ve been provided explicit consent to do so from the participant. This is especially important when discussing personal or sensitive topics.
  • Use direct quotes, audio recordings, or photos from a range of participants to reflect and promote the diversity of your recruit. This provides the fullest picture of who you spoke with during the study.
  • Don’t paraphrase direct quotes nor infer too heavily without solid context or understanding. This maintains impartiality and avoids unconscious bias.
  • Avoid reporting demographic information for individual qualitative findings as it may inadvertently support institutionalized biases or stigma (i.e. 2 male Caucasians said X, 1 black woman said Y). It is best practice to segment by demographics only with quantitative samples where meaningful differences may be found through statistical analysis.

User Personas

A common framework for communicating generative user research findings is the user persona — an archetypical representation of a group of users that exhibit similar attitudes and behaviors. While incredibly powerful for making design decisions for months or even a year after the completion of research, they’re just as powerful in reinforcing gender, age, or racial stereotypes.

Care should be taken in the crafting of user personas so that they uphold their humanizing character while avoiding enforcing bias. While there are other resources for creating agnostic user persons, here are a few highlights:

  • Either avoid using human names altogether. Using human names often promote biased stereotypes given strong societal expectations around different gender roles, positions of authority, or occupational status. Instead, you may lean into pithy 2-3 word titles which better exemplify who your Personas are (i.e. ‘Cautious Comparer,’ or ‘Impulsive Spender’).
  • Use gender-neutral and culturally diverse names. While the above catchy titles are still recommended if you insist on using names in order to humanize the personas, use gender-neutral names to fight the unconscious gender bias. There are also many cultures and backgrounds with which your users may identify — using only names from western European etymology may incidentally erase their unique identity. Using names from diverse etymology helps fight this bias. Here are a few examples of gender-neutral, culturally diverse names: Adrian, Armani, Brett, Devon, Kai, Krishna, Maren, Noor, Nilam, Sam, Jaylin, Jordan, Yoshi.
  • As with the above, leverage artistic visualizations which exemplify the Persona more accurately than a fictitious image or photo. For example, in a Persona for a car manufacturer, it is much more telling to see a weighted scale with a dollar sign on one side and the word ‘safety’ on the other side than it is to see a stock photo of a worried looking person in the front seat. Using photos or illustrations for Personas further stigmatizes race, gender, and body type. If insistent on using humanizing imagery, consider artful illustrations which are ambiguous in gender, race, and body types in order to remain inclusive.
  • Do not include disability status unless this was directly part of your research or you’re creating a set of personas specifically focused on people of varying abilities or with a focus on intersectionality. Consider if this level of information or detail is important for the product you’re creating. While the goal of including this information may be with good intentions, you may be misrepresenting already marginalized communities.
  • Try and avoid gender, race, sexual orientation, or other demographics that are more akin to segmentation analysis and not user personas. Personas should be representations of like-attitudes and especially behaviors which often transcend basic demographics.

As apparent by the above guidance, creating Personas tows a very fine line between creating a useful fictitious archetype and promoting bias and stigma. The latter is highly unethical and leads to false assumptions and even worse design and content strategy decisions.

Design Prototypes

Similar to the above guidance given to User Personas, the same care should be taken with users personified through prototype designs. Often we create situations or scenarios to stress test the content and interaction design of our systems by placing a fictitious character at the heart of our mock-ups.

Consider leveraging the gender-neutral and culturally diverse names above for use within your prototypes (i.e. the account name of an authenticated user). For account or profile images, leverage artistic illustrations, or a diverse range of stock photography to depict your users. When creating designs where multiple individuals will be represented, take a step back to ensure diversity in the holistic group rather than a focus on one gender, race, culture, or body type.

Bringing It All Together

Ethical user research isn’t only the right thing to do but it makes good business sense. Principles of ethical conduct guide our decision-making, keeping us out of trouble and holding us accountable to our users and society.

By influencing your practices, templates, and processes with a foundation of ethical principles you will show your team and your organization that you value the dignity and welfare of your users. Building an ethical user research practice at any organization is not inherently difficult, however, requires upfront and ongoing conscious investment in order to do so effectively.

It’s up to us to put the humans we design for at the center of our own business practices and create a true form of human-computer interaction. No one will do this work for us. Noone will hold us accountable. After all, ethics (especially in our field) are an exclusively human concern with no superhuman authority behind it.

Are you ready to make a difference?

Articles on Smashing Magazine — For Web Designers And Developers

doublediamond-intro
30 May

How To Run The Right Kind Of Research Study With The Double-Diamond Model

by TBSCategories: News

How To Run The Right Kind Of Research Study With The Double-Diamond Model

How To Run The Right Kind Of Research Study With The Double-Diamond Model

Steve Bromley

Product and design teams make a lot of decisions. Early on in the development of a product, they will be thinking about features — such as what the product should do, and how each feature should work. Later on, those decisions become more nuanced — such as ‘what should this button say? Each decision introduces an element of risk — if a bad decision is made, it will reduce the chance for the product to be successful.

The people making these decisions rely on a variety of information sources to improve the quality of their decision This includes intuition, an understanding of the market, as well as an understanding of user behavior. Of these, the most valuable source of information to put evidence behind decisions is understanding our users.

Being armed with an understanding of the appropriate user research methods can be very valuable when developing new products. This article will cover some appropriate methods and advice on when to deploy them.

A Process For Developing Successful Products

The double diamond is a model created by the UK’s Design Council which describes a process for making successful products. It describes taking time to understand a domain, picking the right problem to solve, and then exploring potential ideas in that space. This should prove that the product is solving real problems for users and that the implementation of the product works for users.

Diagram showing the design council’s Double Diamond model
The design council’s Double Diamond model (Large preview)

To succeed at each stage of the process requires understanding some information about your users. Some of the information we might want to understand from users when going through the process is as follows:

The double diamond image with user research questions linked to each phase
Some research questions appropriate for each stage (Large preview)

Each stage has some user research methods that are best suited to uncovering that information. In this article, we’ll refer to the double diamond to highlight the appropriate research method throughout product development.

Diamond 1: Exploring The Problem And Deciding What To Fix

The first diamond describes how to come up with a suitable problem that a new product or feature should fix. It requires understanding what problems users have, and prioritizing them to focus on a high-value area. This avoids the risk of building something that no-one is going to use.

The most effective way of understanding the problem is to get true first-hand experience of users performing real tasks in context. This is best done by applying ethnographic and observational methods to identify the range of problems that exist, then prioritizing them using methods such as surveys.

Double Diamond Phase Appropriate Method Why?
Explore the problem Ethnographic and Observational studies Gives deep insight into what problems people have that can inspire product decisions
Decide what to fix Surveys Discovers how representative problems are, and helps prioritise them

We’ll review each method, in turn, to describe why it’s appropriate.

Explore The Problem With Ethnographic And Observational Methods

The first phase of the double diamond is to ‘explore the problem’. User research can build up an understanding of how people act in the real world and the problems they face. This allows the problem space to be fully explored.

The double diamond image with ‘Explore the problem’ highlighted
Explore the problem (Large preview)

This valuable behavioral information is only uncovered only by watching people do real tasks and asking them questions to uncover their motivations and issues. Doing early qualitative research will help identify the problems that people have. These problems can inspire product ideas, features, and help teams understand how to successfully solve user’s problems. This information will also help disregard poor product ideas by revealing that there is no real need for it. This leads to a more useful product being developed and increasing the chance of success.

The most appropriate methods for doing this are ethnographic. This can include diary studies, where a user’s interaction with the subject matter is captured over a number of weeks. This reveals issues that wouldn’t turn up in a single session or that people wouldn’t remember to talk about in a lab-based interview.

This isn’t the only way of uncovering this kind of in-depth information though. Other suitable observational methods include watching people use existing software or products, both in the lab or in the wild. These are quicker and easier to run than diary studies, but are limited to only capturing a single interaction or what the participant will remember when prompted. For some subject matters, this might be enough (e.g. navigating an online shop can be done and explored in a single session). More complex interactions over time, such as behavior with fitness trackers, would be more sensible to track as a diary study.

Decide What To Fix With Surveys

The second half of the first diamond comes next. Having understood real user’s contexts and what problems they have, these can then be documented and prioritized to ‘decide what to fix’.

The double diamond image with ‘Decide what to fix’ highlighted
Decide what to fix (Large preview)

This prioritization will be done by product managers who take into account many factors, such as “what do we have the technical ability to do” and “what meets our business goals”. However, user research can also add valuable information by uncovering the size of the issues users have. Surveys are a sensible approach for this, informed by the true understanding of user behavior uncovered in the earlier studies. This allows teams to size the uncovered issues and reveal how representative the behaviors discovered are.

Combining quantitative methods with generative user research studies help inspire early decisions about what a product should do. For example, Spotify’s discovery work on how people consume music analyzed primary research fieldwork to create personas and inform their development work. This allows a team to complete the first diamond with a clear understanding of what problem their product is trying to solve.

Diamond 2: Test And Refine Potential Solutions

The second diamond describes how to end up on a successful implementation of a product to fix the problem. Having decided which problem to fix, research can then explore different ways of fixing that problem, and help refine the best method.

Double Diamond Phase Appropriate Method Why?
Test potential solutions Moderated usability testing Creates a deep understanding of why the solution works, to inform iteration
Refine final solution Unmoderated usability testing Can get quick results on small questions, such as with the UI

Test Potential Solutions With Moderated Usability Testing

The second diamond in the double diamond design process starts with evaluating a variety of solutions in order to decide the best possible implementation of a product. To achieve this with rigor requires usability testing — creating representative prototypes and then observing if users can successfully complete tasks using them.

The double diamond image with ‘Test Potential Solutions’ highlighted
Test potential solutions (Large preview)

This kind of study takes time to do properly, and attention on each individual’s user experience to understand what causes the behavior that is observed during usability testing. A moderated session, with the researcher present, can ask probing questions to uncover things that participants won’t articulate unprompted such as “what are you thinking currently” or “ why did you decide to do that?”. These kinds of studies reveal more data when a moderator is able to ask participants these questions, and avoids missing the opportunity to gather more data from each study, which can be used to evaluate and iterate the product. A single moderated research session potentially reveals more useful information than a series of unmoderated tests.

This kind of in-depth exploration of the problem has been a key part of AirBNB’s early success. In 2009 the company was close to bankruptcy and desperate to understand why people were not booking rooms. By spending time with users reviewing the ads on their website, they were able to uncover that the pictures were the problem. This then allowed them to focus their iteration on the process for gathering photos of rooms, which put them on the path for changing hotel booking forever. As the global pandemic changes people’s behavior with holidays in the future, in-depth qualitative research will be essential as they continue to adapt to new challenges.

This doesn’t mean that the moderator has to be in the same room as the participant. Often it can be very valuable to find participants who are geographically remote, and avoid over-sampling people who live in major cities, which is often where research teams are based. Screen sharing software, such as google hangouts or zoom can make remote sessions possible, while still having the session run live with a moderator.

Refine Final Solution With Unmoderated Usability Testing

The final stage of the double diamond describes refining the final solution, which can require a lot of small iterative tests. A shortcut to the deep insight from moderated testing is remote unmoderated research. This includes tools like usertesting.com which allow teams to put their software in front of users with little effort. By sending a website URL to their panel of users, they send back videos of their participants using the website and commenting on their experience.

The double diamond image with ‘Refine final solution’ highlighted
Refine final solution (Large preview)

This method can be popular because it is quick (multiple sessions can run simultaneously without a moderator present) and cheap (participants aren’t paid a huge amount to take part). Because of this, it is often considered an appropriate tool by companies looking to start doing user research, however, care needs to be taken.

This method has constraints which means that it’s only sensible for later on in the design process. Because the participants on these kinds of websites are all people who test multiple websites regularly, they become increasingly different to a normal user. This changes their behavior while using websites and makes it dangerous to draw conclusions from their behavior about what other users would understand or do. This is called a sampling bias — creating a difference between ‘real’ users, and the users being tested.

Because of these risks, these studies may be most appropriate late in development, testing content or UI changes, when the risks of getting decisions wrong are much lower. Iterative studies ensure that users understand what has been created, and are able to use it in the way the designer intended. An example of this is the iterative usability testing the UK’s Government Digital Service ran to ensure citizens could successfully identify themselves and access government services.

After The Double Diamond

As we’ve covered, it is important to select the right method to de-risk product decisions before launch. When a product is launched, it will be immediately obvious whether there is an audience for it, and whether people understand and can use the product — both through how well the product sells, and through reviews and customer feedback.

The double diamond image with ‘Solution delivered’ highlighted
After the double diamond (Large preview)

Nevertheless, launching the right product doesn’t mean that the opportunity for research is over. New opportunities to explore real user behavior will continue to inspire adding or removing features, or changes to how the product works.

Double Diamond Phase Appropriate Method Why?
Solution delivered Analytics + moderated usability testing combined Inform future updates post-launch with qualitative and quantitative insight.

Combining some of the methods we’ve described previously with new data from analytics will continue to drive high-quality decision making.

Research After The Solution Is Delivered With Analytics

Post-launch analytics are an important part of building a complete understanding of the behavior of users.

Analytics will reveal what people are doing on a website. However, this information is most valuable when combined with understanding why that behavior is occurring. It is also important to be aware that analytics are only seeing a short section of a user’s experience, the part that happens on your website and their whole end-to-end journey also includes a lot of things that happened off the site, or in the real world. Building a research strategy that combines insight from analytics with an understanding of motivations from qualitative studies is a powerful combination to inform decision making.

This requires close collaboration between the analytics team and the user research team — regular community events, skills sharing and project updates will create awareness of the priorities of each team, the type of research questions they can support one another with and identify opportunities to work together, leading to a stronger combined team.

Optimize Your Research Process

In this article, we’ve covered some appropriate methods to use to inform product development. However, there can still be resistance to running the right kind of study.

New research teams may be asked to cut corners. This can include suggesting participants who are convenient, such as friends, without taking the time to screen them to ensure they represent real users. This can be suggested by colleagues who are unaware of the risks caused by taking decisions based on unrepresentative research.

In addition to running research studies, a researcher has to be an educator and advocate for running the right kind of studies and help their colleagues understand the differences in quality between the type of information gathered from different research methods. Presentations, roadshows, and creating posters are some techniques that can help achieve this.

Incorporating user research into decision making can be quite radical at some organizations, particularly those with a history of deferring to client wishes or listening to the highest-paid person in the room. A lot of hard work and creativity are needed to bring about change in how people work. This requires understanding the decision maker’s current incentives, and describing the benefits of research in a way that shows how it makes their life easier.

If an organization understands and accepts why running studies using appropriate methods it shows a real desire for improving the quality of decision making within the organization. This is an encouraging sign that a new research team has the potential to be successful.

The next step for new researchers will be to establish the logistics of running research, including creating a research process, building out the tools and software needed, and identifying the highest priority research questions for your organization. There is a lot of great guidance from the research community on techniques to do this, for example, the work being done by the research ops community.

Smashing Editorial (ah, ra, il)


Articles on Smashing Magazine — For Web Designers And Developers

design-systems-large-opt
11 Jul

Better Research, Better Design, Better Results

by TBSCategories: News

Better Research, Better Design, Better Results

Better Research, Better Design, Better Results

Sam Wright & James Macnamara

Over the years, one thing we have consistently seen is how little insight from digital marketers is used at the planning stages of a web development project.

Data from Google Analytics and SEMrush to tools like VWO (Visual Website Optimizer) or Hotjar are all resources that can be used to provide valuable insight ahead of the first line of code being written. Basic SEO elements, such as URL structure and metadata, should also be involved in the decision making of any web design project.

This has been pointed out before, and it’s a sore point for many SEO and content specialists. However, in this article we’re going to focus on the issue in relation to our own preferred methodology, which is effective content research and creation, and how user intent affects the process at every stage.

We’ll then move on through each aspect of the design process, talking about SEO questions along the way, and ending up with a detailed breakdown of a workflow we feel achieves two things: websites which look great, and are fully-realized assets designed to achieve measurable goals.

Intelligent Content Research

A website doesn’t just have to be built. It has to populated with material. The way this material is designed will have a large part in determining a website’s success, i.e. what it brings to a client’s business or organization.

This is why we find it strange that a normal web design process misses out at its earliest stages things like keyword research, and its more developed relative — content strategy. So often a frame is built without enough thought about what it’s going to contain.

All of our projects at some level require keyword research, and this always involves careful attention to user intent. As SmashingMag readers, you’ll most likely understand this concept. For the sake of clarity though, it is worth revisiting this in terms of content strategy and SEO.

Before user intent was a thing, keyword research involved gathering lists of search volumes and “difficulty” numbers and trying to spot what keywords you might rank for, without too much attention paid to whether they were queries actually likely to be used by your ideal users.

While we still have to go through this process, effective research requires more intelligent use of the data we find. We have to focus on discovering target keywords and developing material that satisfies the intent behind the query — while still looking out for some relevant “good opportunity” keywords (i.e. high volume, low competition) along the way.

This means that keyword research is becoming a way of understanding what users mean by their searches in context, what questions they want answering, and what kind of language they use, all serving the purpose of creating content that has the best chance of helping a website meet its owner’s goals.

User Intent And Content Creation

User intent informs keyword research, which in time becomes content strategy, and then creation. The content we create always has a purpose, and in the majority of cases it is to satisfy the intent behind a user query.

As a broad example, let’s take the query “coffee”. Here’s how the results look — notice the different types of content aimed at meeting varying intents:

Search engine results page for coffee
(Large preview)

The results vary hugely according to the audience they are targeting. Some are aimed at people wanting to find somewhere to grab a coffee nearby, others are sites where you can order your joe online. There are also resources looking at coffee’s history and nutritional information.

While we don’t often have to deal with such broad terms, all of this has to be thought about, unpicked and planned according to a website’s purpose. This means content research, when focused on users, has obvious and enormous implications when it comes to site architecture and even aesthetics — i.e. the first things to be worked out in any design process.

When Content Isn’t Considered

One of the most common issues we see with both old and new sites is content that has not designed to fully address user queries, in terms of exact phrases as well as general intent. In some cases, this is easy to fix — for example, a few tweaks to a page’s metadata and copy can often clarify its query and user targeting almost instantly.

In many others though, the problems are much more serious, and a revised architecture or navigation is required as part of an entirely new content strategy — a costly process that could have been avoided if the right professionals had been consulted all along.

Here are some scenarios specific to site content we’ve encountered too many times:

Scenario 1: Shiny New Website, Dull New Content

A client — let’s call him John — is launching a completely new site, with no previous content to refer to.

However, if John isn’t prompted to think about copy, content or SEO until much later down the line — typically after the back-end development phase — then poor decisions can be made, while there is also the risk that he will lose some of his motivation, energy, and patience with the project.

A rush to see it completed means the content isn’t researched or executed well enough to be effective in the long term. Eventually it has to be looked at again during a lengthy and costly second-stage SEO and content creation campaign.

Scenario 2: Same Content, Same Problems

A rebuild of an existing site means there’s existing content to look at and refer to. Sometimes, John is so rushed, or is so intent on keeping costs down at this stage, content is not considered at all.

The same content is used on the old site as on the new site, and John wonders why his site doesn’t shoot immediately to number one for all of his top keywords. Eventually it has to be looked at again during a lengthy and costly second-stage SEO and content creation campaign.

New Content, Or Else!

Sometimes a valuable, authoritative site is rebuilt, as part of a rebrand for example. John insists that everything is new. Without the proper research spelling this out, for example analytics data (explored in more detail below), John isn’t aware of the assets he already has. He gets rid of the old content (or does something even worse like switch to a new domain) that search engines thought was valuable, and rankings mysteriously tank. Eventually it has to be looked at again during a lengthy and costly second-stage SEO and content creation campaign.

Workflow Issues When SEOs Are Called In After The Fact

We have to make do with what we get of course, but it is frustrating for SEOs to work on projects well after problems have set it, and we end up having to suggest that a relatively new site needs to be pulled apart if it has any hope of realizing its value.

When SEO isn’t considered from the beginning, the page layout and semantic markup haven’t considered excerpts, H-tags, metadata or how the CMS can aid SEO in the long term. Many clients will then turn to quick fixes such WordPress plugins like Yoast. There’s a good chance that these will be ineffective or used incorrectly, perpetuating the problems at hand.

Graph showing negative results
(Large preview)

So, guess what? An SEO specialist is brought in after the site has been launched.

Now the client is unhappy with their existing agency and places a high importance on improving SEO. In turn, the SEO specialist has a difficult job trying not to undermine the web agency but still needs to recommend structural and on-page adjustments.

They will also face problems with client expectations, who will unsurprisingly feel ripped-off and begrudge spending more money on their new shiny website.

Does this all sound familiar? The crux of our argument is that by bringing inline processes such as intent-focused keyword research from the beginning, these situations can be avoided and everyone can get along.

At the same time, an integrated approach will mean better UX and conversions alongside a strong SEO performance. Better focused content can also mean lower PPC costs too, as relevancy is a part of Google’s Adword calculation.

Rather than an expensive design phase followed by a second round of expensive SEO work, the whole process can be streamlined, keeping time and costs down, clients happier, and producing a better final product as a result.

A New Design Process

This is all well and good, but how can we put it into practice? With varying degrees of complexity, for many in the industry the design process will look like this:

A Typical Workflow

Flow graph showing the different processes, project planning, design, and development & launch
(Large preview)

It’s worth stating that good developers will focus on user experience and the visitor journey in their own workflow. Instead, a typical project may move through these stages:

Flow graph showing the different processes, project planning, design, and development & launch
(Large preview)

A Different Approach

Over the past year or so, we have put a lot of effort into refining this process in a way that we believe gives the best possible value for our clients. Here it is:

Project planning
Flow graph showing the different processes, specifically project planning and defining budget
(Large preview)

As always, this should be the first step, as it will define the scope of the work ahead. Be realistic and build in room for error, and be very aware that you get what you pay for. Under-budgeting runs the risk of falling short on key areas such as design, functionality, and content. At the same time, if all the project’s budget is swallowed up on design and development, there will be no room for a supporting marketing strategy or ongoing updates and improvements.

Flow graph showing the different processes, specifically project planning and defining goals
(Large preview)

Similarly, your goals should be clear from the very start. Are you focused on acquiring email addresses or selling products? What is the one thing that you want your visitors to do above all else? Without clearly understanding this, the chances are that your site will fall short in its aims.

Once this is decided, you can move on to setting broader targets. There are a number of methods here, such as SMART goals (or Specific, Measurable, Attainable, Relevant and Time Bound). These will define how a successful project will look on completion. Be realistic here — if your current site has a few hundred visits per month, don’t expect this to reach 10,000 within a few months without some serious effort and investment.

Sales website goals examples, including generating more sales, improving sales conversion rate, and improving sales support
(Large preview)

At the same time, we are big fans of the Objectives and Key Results (OKR) approach that is used by Google, LinkedIn et al. This technique can work really well for a web project as well a general business strategy.

Here’s a great video that will give you some background on the OKR system.

Writing effective OKRs is a bit of an art in itself, but there are some good examples here. The main thing to remember though is that your goals are going to define the site’s architecture to some extent.

At its simplest level, people won’t be able to contact you if there is no contact form. Similarly, they will be less likely to get in touch if you remove a bunch of FAQ or blog posts that help explain what it is that your product or service does. This brings us onto our next step.

Flow graph showing the different processes, specifically project planning and reviewing existing Google Analytics data, if available
(Large preview)

You may have pages that are already performing well. If that is the case, you’ll need to identify them so you can make sure they are built into your new structure. If you shed pages that bring in traffic at any point of your funnel, this could result in a loss of leads or sales. Along with URL alterations, this can be one of the main causes of drops in traffic after a migration or significant site update. It may seem obvious but it is an issue that we’ve seen time and time again.

The first stage of preventing this is to look in Google Analytics, or whichever analytics platform you use. Find out which pages are bringing in organic visits first of all. These should be built into your new plan as a priority, preferably without changing the URL and keeping a prominent place in your navigation structure.

Another great tool here is Keyword Hero. This is relatively new, but it plugs into Google Analytics and removes the <not provided> tag that was applied to organic keywords a few years ago.

Example of the not provided tag
(Large preview)

This uses some clever machine learning, and it means that you’ll be able to see which keywords drive traffic to specific pages on your site. This is extremely useful in terms of planning which pages to keep or remove.

Of course, not all pages are important in terms of organic traffic. As mentioned, some could be crucial pre-conversion or sale, such as an FAQ page, but bring in little of the may of inbound visits to the site. Take a look at page views, and user flow here to make sure you are not missing anything.

At the same time, it’s worth bearing in mind that your data may not be perfect. Checking the validity of Google Analytics data is a pretty big subject in itself, but one of the simplest steps you can take is to check that your tracking code is implemented correctly.

Again, we won’t go into the ins-and-outs here. However, there is one trick that we recommend when carrying out content migrations. The web crawler Screaming Frog has a nifty feature that allows you to check for Analytics code on every page. More than once, we’ve uncovered valuable pages that are not being tracked, and would have been lost in a redesign.

Flow graph showing the different processes, specifically project planning and reviewing current rankings (keywords and ranking pages), if available
(Large preview)

Next, it’s time to start looking to see which keywords you are visible for. There are a few tools we use here, but the most useful is SEMrush. This monitors billions of keywords and tracks which sites are ranking for them. By querying its database, you can see which keywords your site is appearing for in Google’s results without manually entering them into a tracker. It’s by no means perfect, so you’ll need to manually check positions for any terms you think it may have missed too.

Example of top organic keywords from SEMRUSH
(Large preview)

Once you have this information, you can start drawing it together in a spreadsheet. Here is a sample document, and you can see the initial findings in the first tab.

Example of a keyword research document
(Large preview)
Flow graph showing the different processes, specifically project planning and creating audience profiles
(Large preview)

For both UX and SEO, it is important to understand who you are speaking to. Think about the types of language or phrases your users will know, as well as the tone of voice. Do they respond to images or copy, detail or bullets, flashy designs or more technical pages?

Keyword research is also really useful here, as it defines terms and reveals correct vocabulary — another example of how keyword research eventually filters down and is important to almost every step.

low graph showing the different processes, specifically project planning and carrying out user-intent focused keyword research
(Large preview)

Now that we know who we are talking to, how best can you do it? We have explained the concept behind user-intent focused keyword research earlier in this document, but here’s some insight into how we go about doing it ourselves. Please note, this could be a feature in itself, so for the sake of brevity we’re just focusing on an outline here.

In terms of our toolkit, we tend to use a combination of SEMrush and Moz. We feel that using both, as well as AdWords’ keyword planner, and any others you can get your hands on, is the best way of gathering data, as each tool will have its own strength, and often data for longer-tail keywords will be available in one tool, but not another.

Here are the first steps.

  • Listing all the relevant keywords we can find along with the data we have for them, volume being the most important.
  • We’ll also include some measure of how competitive they are, as well an indication if the current is already ranking for them. We usually use Moz data here, which corresponds to this key.
Key
0 – 15% Non-competitive term, top rankings achievable with well optimized on-page keyword use
16 – 30% Low competition, top rankings achievable with well optimized on-page keyword use and light link strength
31 – 45% Slightly competitive, top rankings require well optimized on-page use and moderate link strength
46 – 60% Competitive, top rankings achievable only with highly optimized on-page content and substantial link strength
61 – 75% Highly competitive term, top rankings require on-page optimization, well-established history and robust link strength
76 – 90% Exceptionally competitive term, top rankings only achievable with highly-established site and overwhelming link strength
91%+ Among the most competitive terms on the web, only the most powerful & popular sites can achieve rankings
  • We gather as much as possible here, so the client can see the research for themselves and so we can see everything at once — of course, most of it won’t end up being used, but long lists look more thorough than a few simple, if well-researched proposals.

It’s what you do with the data you gather that makes the research different and far more valuable than say, five years ago, when user intent wasn’t so important to or understood by search engine algorithms.

From keyword research data the site structure and list of pages needs to emerge, and be thought of as intelligently as possible. To this end:

  • We look through everything we find and select keywords based on volume, competition, but most importantly of all, whether the site will be able to effectively meet the user intent behind the query. Sometimes the numbers just click together, but mostly you’ll have to compromise — with user intent always being the most important consideration.

  • We then use the most general or short-tail keywords we select and think of them as intent or topic “nodes” in order to deepen our research and increase our insight into potentially valuable content.

As well as looking at keywords focused on landing pages, needs and wants keywords and exact phrases (for example questions that are also verbatim search queries) are also crucial. AnswerThePublic is a great tool for branching out and seeing what users are wondering about your chosen topics/keywords.

Example of a result from Answer The Public
(Large preview)
  • By branching out, you discover new users with new intent, and think of new content to meet them. The site is built catering to more users as a result, it ranks for more queries, it gets more traffic, its authority grows and you end up with a virtuous circle — as opposed to the vicious cycle we had before.

With well-researched content present when it launches, the site is able to realize its value from day one, so the client ends up with more conversions, more revenue. This way, the extra costs involved during the site build are more than offset.

Flow graph showing the different processes, specifically project planning and creating a sitemap and architecture
(Large preview)

With all this information, it’s time to start planning out the site. Define what goes on what page. Understand where the content is going on the website AND WHY. Make it scalable — adding or removing content should be easy as business goals can rapidly shift.

For this stage, everything needs to make sense. Pages need to be linked because it makes sense semantically. Those that are important for both users and search engines should be high up in your navigation.

E-commerce sites often do this well. Take the example below — the category and sub-category structure means that it is clear keywords should be used for the page.

Example of a menu from B&Q
(Large preview)

On the other hand, here is an example of a site where the navigation is a wasted opportunity.

Example of a menu from Ziggurat
(Large preview)

There are no services pages that could target keyword groups, and no sub-pages off any of the main categories. While “minivation” may well be a great concept, it’s not one that users will search for. Of course, this may not be a priority in this instance, but we see this kind of layout time and time again.

Overall, the danger here can be that without an awareness of SEO at this stage, the client can want to switch from a navigation like our first example to the second. In this case, there is an enormous risk to traffic and therefore revenue, and as web professionals, it is our duty to state this clearly.

Flow graph showing the different processes, specifically project planning and targeted copywriting/marketing messaging
(Large preview)

While content production usually happens at the end of a project, we feel designing around real content (rather than lorem ipsum) is more cost and time efficient, as it greatly reduces the need for design amendments after a project is complete.

There is also a really strong case here that placeholder text reiterates the idea that content is secondary to design, and that it is something lesser in the hierarchy of the project. This is an idea that again has been covered brilliantly by Kyle Fiedler, so there is no need for us to tread over the same ground.

At the same time, by this point, your research will give you all the information you need to put together an amazing brief for your writers. Believe us, they’ll appreciate it!

Design
Flow graph showing the different processes, specifically design and low fidelity wireframes
(Large preview)

It’s time to start bringing it all together. Initial wireframes should be basic boxes and titles defined by the content development and copy generated up until this point, outlining key sections of the website. Again, wireframe with real content wherever possible. Tools like Balsamiq and wireframe.cc are really useful for this.

Flow graph showing the different processes, specifically design and high fidelity wireframes
(Large preview)

Once the wireframes are created the designs can start becoming more realized. Add in some brand identity, such as a color pallet, the actual client logo, corporate typography, and fonts. At this point, you should start to see exactly how the website will look. Any changes should be made at this stage — it’s much easier to edit a Photoshop file than change code.

Development

By this stage, the actual development phase should be straightforward. Write the HTML and CSS code for the basic design, then focus on any interactive elements. From an SEO point of view, it is worth stating that Javascript is a pretty hot topic. Google is far from perfect at handling JS, so scripts that control the display of navigations or key content need to be implemented very carefully. More on this topic can be found here.

Flow graph showing the different processes, specifically development and content population
(Large preview)

In our experience, this often is the slowest part of any project. However, with all of the content creation finished early on in the process, this task should just require a copy-and-paste into the CMS, saving considerable time, stress and delays.

Flow graph showing the different processes, specifically design and rounds of testing
(Large preview)

As usual, test, test, and test again. Crawl the site, add all of your tracking codes, add to Search Console, make sure it’s indexed — the full works!

Flow graph showing the different processes, specifically development and launch, and post launch review
(Large preview)

Were we right? Have the goals succeeded? A website is never finished. Keep tracking and reporting, always remembering the goals set out at the start of the project.

Although it might seem a lot, only a few extra steps have been added to the whole process. With keyword research and a content strategy the focus at the start of the project, the aims of the site are more clearly defined, and its entire structure mapped out and understood, with everything in its right place. Two costly and complex projects, an SEO/content campaign and web design, become one — and one that is far more manageable, efficient, and ultimately produces a better result.

This is kind of an ideal scenario — most of the time our work involves working on sites that have been built without SEO in mind, and we come to help afterwards. We see our roles shifting as more people realize the logic behind SEOs, developers, and designers working together on projects, rather than in sequence, undermining each other’s efforts along the way.

Further Reading

Smashing Editorial (ra, il)


Articles on Smashing Magazine — For Web Designers And Developers

design-systems-large-opt-2
04 May

FAST UX Research: An Easier Way To Engage Stakeholders And Speed Up The Research Process

by TBSCategories: News

FAST UX Research: An Easier Way To Engage Stakeholders And Speed Up The Research Process

FAST UX Research: An Easier Way To Engage Stakeholders And Speed Up The Research Process

Zoe Dimov

Today, UX research has earned wide recognition as an essential part of product and service design. However, UX professionals still seem to be facing two big problems when it comes to UX research: A lack of engagement from the team and stakeholders as well as the pressure to constantly reduce the time for research.

In this article, I’ll take a closer look at each of these challenges and propose a new approach known as ‘FAST UX’ in order to solve them. This is a simple but powerful tool that you can use to speed up UX research and turn stakeholders into active champions of the process.

Contrary to what you might think, speeding up the research process (in both the short and long term) requires effective collaboration, rather than you going away and soldiering on by yourself.

The acronym FAST (Focus, Attend, Summarise, Translate) wraps up a number of techniques and ideas that make the UX process more transparent, fun, and collaborative. I also describe a 5-day project with a central UK government department that shows you how the model can be put into practice.

The article is relevant for UX professionals and the people who work with them, including product owners, engineers, business analysts, scrum masters, marketing and sales professionals.

1. Lack Of Engagement Of The Team And Stakeholders

“Stakeholders have the capacity for being your worst nightmare and your best collaborator.”

— UIE (2017)

As UX researchers, we need to ensure that “everyone in our team understands the end users with the same empathy, accuracy and depth as we do.” It has been shown that there is no better alternative to increasing empathy than involving stakeholders to actually experience the whole process themselves: from the design of the study (objectives, research questions), to recruitment, set up, fieldwork, analysis and the final presentation.

Anyone who has tried to do this knows that it can be extremely difficult to organize and get stakeholders to participate in research. There are two main reasons for this:

  1. Research is somebody else’s job.
    In my experience, UX professionals are often hired to “do the UX” for a company or organization. Even though the title of “Lead UX Researcher” sounds great and very important in my head, it often leads to misconceptions during kick-off meetings. Everyone automatically assumes that research is solely MY responsibility. It’s no wonder that stakeholders don’t want to get involved in the project. They assume research is my and nobody else’s job.
  2. UX process frameworks are incomplete.
    The problem is that even when stakeholders want to engage and participate in UX, they still do not know *how* they should get involved and *what* they should do. We spend a lot of time selling a UX process and research frameworks that are useful but ultimately incomplete — they do not explain how non-researchers can get involved in the research process.
Problems associated with stakeholders involvement in UX Research.
Fig. 1. Despite our enthusiasm as researchers, stakeholders often don’t understand how to get involved with the research process.

Further, a lot of stakeholders can find words such as ‘design,’ ‘analysis’ or ‘fieldwork’ intimidating or irrelevant to what they do. In fact, “UX is rife with jargon that can be off-putting to people from other fields.” In some situations, terms are familiar but mean something completely different, e.g., research in UX versus marketing research.

2. Pressure To Constantly Reduce The Time For Research

Another issue is that there is a constantly growing pressure to speed up the UX process and reduce the time spent on research. I cannot count the number of times when a project manager asked me to shorten a study even further by skipping the analysis stage or the kick-off sessions.

While previously you could spend weeks on research, a 5-day research cycle is increasingly becoming the norm. In fact, the book Sprint describes how research can dwindle to just a day (from an overall 5-day cycle).

Considering this, there is a LOT of pressure on UX researchers to deliver fast, without compromising the quality of the study. The difficulty increases when there are multiple stakeholders, each with their own opinions, demands, views, assumptions, and priorities.

The Fast UX Approach

Contrary to what you might think, reducing the time it takes to do UX research does not mean that you need to soldier on by yourself. I have done this and it only works in the short term. It does not matter how amazing the findings are — there is not enough PowerPoint slides in the world to convince a team of the urgency to take action if they have not been on the research journey themselves.

In the long term, the more actively engaged your team and stakeholders are in the research, the more empowered they will feel and the more willing they will be to take action. Productive collaboration also means that you can move together at a quicker pace and speed up the whole research process.

The FAST UX Research framework (see Fig. 2 below) is a tool to truly engage team members and stakeholders in a way that turns them into active advocates and champions of the research process. It shows non-researchers when and how they should get involved in UX Research.

The FAST UX Research approach; FAST UX Research methodology.
Fig. 2. The FAST User Experience Research framework

In essence, stakeholders take ownership of each of the UX research stages by carrying out the four activities, each corresponding to its research stage.

Working together reduces the time it takes for UX Research. The true benefit of the approach, however, is that, in the long term, it takes less and less time for the business to take action based on research findings as people become true advocates of user-centricity and the research process.

This approach can be applied to any qualitative research method and with any team. For example, you can carry out FAST usability testing, FAST interviews, FAST ethnography, and so on. In order to be effective, you will need to explain this approach to your stakeholders from the start. Talk them through the framework, explaining each stage. Emphasize that this is what EVERYONE does, that it’s their work as much as the UX researcher’s job, and that it’s only successful if everyone is involved throughout the process.

Stage 1: Focus (Define A Common Goal)

There is a uniform consensus within UX that a research project should start by defining its purpose: why is this research done and how will the results be acted upon?

Focus in FAST UX Research; first stage in the FAST UX Research process.
Fig. 3. Focus is about defining clear objectives and goals for the research and it’s ultimately the team’s and all stakeholders’ shared responsibility to do this.

Generally, this is expressed within the research goals, objectives, research questions and/or hypotheses. Most projects start with a kick-off meeting where those are either discussed (based on an available brief) or are defined during the meeting.

The most regular problem with kick-off sessions like these is that stakeholders come up with too many things they want to learn from a study. The way to turn the situation around is to assign a specific task to your immediate team (other UX professionals you work with) and stakeholders (key decision makers): they will help focus the study from the beginning.

The way they will do that is by working together through the following steps:

  1. Identify as a group the current challenges and problems.
    Ask someone to take notes on a shared document; alternatively, ask everyone to participate and write on sticky notes which are then displayed on a “project wall” for everyone to see.
  2. Identify the potential objectives and questions for a research study.
    Do this the same way you did the previous step. You don’t need to commit to anything yet.
  3. Prioritize.
    Ask the team to order the objectives and questions, starting with the most important ones.
  4. Reword and rephrase.
    Look at the top 3 questions and objectives. Are they too broad or narrow? Could they be reworded so it’s clearer what is the focus of the study? Are they feasible? Do you need to split or merge objectives and questions?
  5. Commit to be flexible.
    Agree on the top 1-2 objectives and ensure that you have agreement from everyone that this is what you will focus on.

Here are some questions you can ask to help your stakeholders and team to get to the focus of the study faster:

  • From the objectives we have recognized, what is most important?
  • What does success look like?
  • If we only learn one thing, which one would be the most important one?

Your role during the process is to provide expertise to determine if:

  • The identified objectives and questions are feasible for a single study;
  • Help with the wording of objectives and questions;
  • Design the study (including selecting a methodology) after the focus has been identified.

At first sight, the Focus and Attend (next stages) activities might be familiar as you are already carrying out a kick-off meeting and inviting stakeholders to attend research sessions.

However, adopting a FAST approach means that your stakeholders have as much ownership as you do during the research process because work is shared and co-owned. Reiterate that the process is collaborative and at the end of the session, emphasize that agreeing on clear research objectives is not easy. Remind everybody that having a shared focus is already better than what many teams start with.

Finally, remind the team and your stakeholders what they need to do during the rest of the process.

Stage 2: Attend (Immerse The Team Deeply In The Research Process)

Seeing first hand the experience of someone using a product or service is so rich that there is no substitute for it. This is why getting stakeholders to observe user research is still considered one of the best and most powerful ways to engage the team.

Attend in FAST UX Research; second stage in FAST UX Research.
Fig. 4. Attend in FAST UX Research is about encouraging the team and stakeholders to be present at all research sessions, but also to be actively engaged with the research.

What often happens is that observers join in on the day of the research study and then they spend the time plastered to their laptops and mobile phones. What is worse, some stakeholders often talk to the note-taker and distract the rest of the design team who need to observe the sessions.

This is why it is just as important that you get the team to interact with the research. The following activities allow the team to immerse themselves in the research session. You can ask stakeholders to:

  • Ask questions during the session through a dedicated live chat (e.g. Slack, Google Hangouts, Skype);
  • Take notes on sticky notes;
  • Summarize observations for everyone (see next stage).

Assign one person per session for each of these activities. Have one “live chat manager,” one “note-taker,” and one “observer” who will sum up the session afterwards.

Rotate people for the next session.

Before the session, it’s useful to walk observers through the ‘ground rules’ very briefly. You can have a poster similar to the one GDS developed that will help you do this and remind the team of their role during the study (see Fig. 3 above).

An observation poster; user research poster.
Fig. 5. A poster can be hanged in the observation room and used to remind the team and stakeholders what their responsibilities are and the ground rules during observation.

Farrell (2017) provides more detail on effective ways for stakeholders to take notes together. When you have multiple stakeholders and it’s not feasible for them to physically attend a field visit (e.g. on the street, in an office, at the home of the participant), you could stream the session to an observation room.

Stage 3: Summarize (Analysis For Non-Researchers)

I am a strong supporter of the idea that analysis starts the moment fieldwork begins. During the very first research session, you start looking for patterns and interpretation of what the data you have means.

Summarize in FAST UX Research; the third stage in FAST UX Research.
Fig. 6. Summarize in FAST UX Research is about asking the team and your stakeholders to tell you about what they thought were the most interesting aspects of user research.

Even after the first session (but typically towards the end of fieldwork) you can carry out collaborative analysis: a fun and productive way that ensures that you have everyone participating in one of the most important stages of research.

The collaborative analysis session is an activity where you provide an opportunity for everyone to be heard and create a shared understanding of the research.

Since you’re including other experts’ perspectives, you’re increasing the chances to identify more objective and relevant insights, and also for stakeholders to act upon the results of the study.

Even though ‘analysis’ is an essential part of any research project, a lot of stakeholders get scared by the word. The activity sounds very academic and complex. This is why at the end of each research session, research day, or the study as a whole, the role of your stakeholders and immediate team is to summarize their observations. Summarizing may sound superfluous but is an important part of the analysis stage; this is essentially what we do during “Downloading” sessions.

Listening to someone’s summary provides you with an opportunity to understand:

  • What they paid attention to;
  • What is important for them;
  • Their interpretation of the event.

Summary At The End Of Each Session

You do this by reminding everyone at the beginning of the session that at the end you will enter the room and ask them to summarize their observations and recommendations.

You then end the session by asking each stakeholder the following:

  • What were their key observations (see also Fig. 3)?
    • What happened during the session?
    • Were there any major difficulties for the participant?
    • What were the things that worked well?
  • Was there anything that surprised them?

This will make the team more attentive during the session, as they know that they will need to sum it up at the end. It will also help them to internalize the observations (and later, transition more easily to findings).

This is also the time to consistently share with your team what you think stands out from the study so far. Avoid the temptation to do a ‘big reveal’ at the end. It’s better if outcomes are told to stakeholders many times.

On multiple occasions, research has given me great outcomes. Instead of sharing them regularly, I keep them to myself until the final report. It doesn’t work well. A big reveal at the end leads to bewildered stakeholders who often cannot jump from observations to insights as quickly. As a result, there is either stubborn pushback or indifferent shrugs.

Summary At The End Of The Day

A summary of the event or the day can then naturally transition into a collaborative analysis session. Your job is to moderate the session.

The job of your stakeholders is to summarize the events of the day and the final results. Ask a volunteer to talk the group through what happened during the day. Other stakeholders can then add to these observations.

Summary At The End Of The Study

After the analysis is done, ask one or two stakeholders to summarize the study. Make sure they cover why we did research, what happened during the study and what are the primary findings. They can also do this by walking through the project wall (if you have one).

It’s very difficult not to talk about your research and leave someone else to do it. But it’s worth it. No matter how much you’re itching to do this yourself — don’t! It’s a great opportunity for people to internalize research and become comfortable with the process. This is one of the key moments to turn stakeholders into active advocates of user research.

At the end of this stage, you should have 5-7 findings that capture the study.

Stage 4: Translate (Make Stakeholders Active Champions Of The Solution)

“Research doesn’t have a value unless it results in decisions and actions.”

—Lang and Howell (2017).

Even when you agree with the findings, stakeholders might still disagree about what the research means or lack commitment to take further action. This is why after summarizing, ask your stakeholders to work with you and identify the “Now what?” or what it all means for the organization, product, service, team and/or individually for each one of them.

Translate in FAST UX Research; the fourth stage in FAST UX Research.
Fig. 7. Translate in FAST UX Research is about asking the team or individual stakeholders to discuss each of the findings and articulate how it will impact the business, the service, and product or their work.

Traditionally, it was the UX researchers’ job to write clear, precise, descriptive findings, and actionable recommendations. However, if the team and stakeholders are not part of identifying actionable recommendations, they might be resistant towards change in future.

To prevent later pushback, ask stakeholders to identify the “Now what?” (also referred to as ‘actionable recommendations’). Together, you’ll be able to identify how the insights and findings will:

  • Affect the business and what needs to be done now;
  • Affect the product/service and what changes do we need to make;
  • Affect people individually and the actions they need to take;
  • Lead to potential problems and challenges and their solutions;
  • Help solve problems or identify potential solutions.

Stakeholders and the team can translate the findings at the end of a collaborative analysis session.

If you decide to separate the activities and conduct a meeting in which the only focus is on actionable recommendations, then consider the following format:

  1. Briefly talk through the 5-7 main findings from the study (as a refresher if this stage is done separately from the analysis session or with other stakeholders).
  2. Split the group into teams and ask them to work on one finding/problem at a time.
  3. Ask them to list as many ways they see the finding affecting them.
  4. Ask one person from each group to present the findings back to the team.
  5. Ask one/two final stakeholders to summarize the whole study, together with the methods, findings, and recommendations.

Later, you can have multiple similar workshops; this is how you get to engage different departments from the organization.

Fast UX In Practice

An excellent example of a FAST UX Research approach in practice is a project I was hired to carry out for a central UK government department. The ultimate goal of the project was to identify user requirements for a very complex internal system.

At first sight, this was a very challenging project because:

  • There was no time to get to know the department or the client.
    Usually, I would have at least a week or two to get to know the client, their needs, opinions, internal pressures, and challenges. For this project, I had to start work on Monday with a team I had never met; in a building I had never worked, in a domain I knew little about, and finish on Friday the same week.
  • The system was very complex and required intense research.
    The internal system and the nature of work were very complex; this required gathering data with at least a few research methods (for triangulation).
  • This was the first time the team had worked with a UX Researcher.
    The stakeholders were primarily IT specialists. However, I was lucky that they were very keen and enthusiastic to be involved in the project and get their hands dirty.
  • Stakeholder availability.
    As is the case on many other projects, all stakeholders were extremely busy as they had their own work on top of the project. Nonetheless, we made it work, even if it meant meeting over lunch, or for a 15-minute wrap up before we went home.
  • There were internal pressures and challenges.
    As with any department and huge organization, there were a number of internal pressures and challenges. Some of them I expected (e.g. legacy systems, slow pace of change) but some I had no clue about when I started.
  • We had to coordinate work with external teams.
    An additional challenge was the need to work with and coordinate efforts with external teams at another UK department.

Despite all of these challenges, this was one of the most enjoyable projects I have worked on because of the tight collaboration initiated by the FAST approach.

The project consisted of:

  • 1 day of kick-off sessions and getting to know the team
  • 2,5 days of contextual inquiries and shadowing of internal team members,
  • Half a day for a co-creation workshop, and
  • 1 day for analysis and results reporting.

In the process, I gathered data from 20+ employees, had 16+ hours of observations, 300+ photos and about 100 pages of notes. Here is a great example of cramming in 3 weeks’ worth of work into a mere 5-day research cycle. More importantly, people in the department were really excited about the process.

Here is how we did it using a FAST UX Research approach:

  • Focus
    At the beginning of the project, the two key stakeholders identified what the focus of research would be while my role was mainly to help with prioritizing the objectives, tweak the research questions and check for feasibility. In this sense, I listened and mainly asked questions, interjecting occasionally with examples from previous projects or options that helped to adjust our approach.

    While I wrote the main discussion guide for the contextual inquiries and shadowing sessions, we sat together with the primary team to discuss and design the co-creation workshop with internal users of the system.

  • Attend
    During the workshop one of the stakeholders moderated half of the session, while the other took notes and observed closely the participants. It was a huge success internally, as stakeholders felt there was better visibility for their efforts to modernize the department, while employees felt listened to and involved in the research.
  • Summarize
    Immediately after the workshop, we sat together with the stakeholders for a 30-minute meeting where I had them summarize their observations.

    As a result of the shadowing, contextual inquiries and co-creation workshop, we were able to identify 60+ issues and problems with the internal system (with regards to integration, functionality, and usability), all captured in six high-level findings.

  • Translate
    Later, we discussed with the team how each of the six major findings translated to a change or implication for the department, the internal system, as well as collaboration with other departments.

We were so perfectly aligned with the team that when we had to talk about our work in front of another UK government department, I could let the stakeholders talk about the process and our progress.

My final task (over two additional days) was to document all of the findings in a research report. This was necessary as a knowledge repository because I had to move onto other projects.

With a more traditional approach, the project could have easily spanned 3 weeks. More importantly, quickly understanding individual and team pressures and challenges were the keys to the success of the new system. This could not have happened within the allocated time without a collaborative approach.

A FAST UX approach resulted in tight collaboration, strong co-ownership and a shared sense of progress; all of those allowed to shorten the time of the project, but also to instill a feeling of excitement about the UX research process.

Have You Tried It Out Already?

While UX research becomes ever more popular, gone are the days when we could soldier on by ourselves and only consult stakeholders at the end.

Mastering our craft as UX researchers means engaging others within the process and being articulate, clear, and transparent about our work. The FAST approach is a simple model that shows how to engage non-researchers with the research process. Reducing the time it takes to do research, both in the short (i.e. the study itself) and long term (i.e. using the research results), is a strategic advantage for the researcher, team, and the business as a whole.

Would you like to improve your efficiency and turn stakeholders into user research advocates? Go and try it out. You can then share your stories and advice here.

I would love to hear your comments, suggestion and any feedback you care to share! If you have tried it out already, do you have success stories you want to share? Be as open as you can — what worked well, and what didn’t? As with all other things UX, it’s most fun if we learn together as a team.

Smashing Editorial (cc, ra, il)


Articles on Smashing Magazine — For Web Designers And Developers

finding-research-participants-g-recruiting
04 Apr

Finding UX Research Participants

by TBSCategories: News

Finding UX Research Participants

For UX designers and design teams, research with stakeholders and users is critical. However, accessing research participants isn’t as easy as it sounds. For both professional and amateur researchers finding people to participate in studies can be an elusive task. We often hear about studies and their findings, but we don’t hear as often how researchers recruit study participants.

Researchers can choose from a variety ways to find participants. Many factors determine the best method to use. This includes resources such as time and money, the research method you’re using, the type or characteristics of participants you want to recruit, and the accessibility of these types of participants. In this post, I’ll remove some of the mystery and provide guidance to those interested in recruiting participants for qualitative UX studies.

Potential research participants are everywhere, if you know what to look for.
Potential research participants are everywhere, if you know what to look for.

Incentives

You can use incentives to increase the likelihood of participation in any of these methods of recruitment. Use of incentives is usually a personal choice. Do you feel incentivized participants provide skewed or biased data? I don’t have any issues with providing incentives. An incentive can be a small token of appreciation ($ 5 gift card) or something more substantial ($ 200 or more depending on time needed and type of participant.

I’ve provided guidance for each method based on my experience with incentives.

Identifying And Interviewing Key Internal Stakeholders

You gain insight when you interview key colleagues, clients, and other relevant stakeholders of a project. Particularly at the beginning of a project. This is a great opportunity to understand everyone’s role, what their vision and hopes are for a project or product, and how you might incorporate their experience into the rest of the project. You can increase buy-in and make people feel like part of the process by including stakeholder interviews in any project. I use the term internal stakeholder broadly to describe individuals who have a vested interest in a product or project who are connected to your organization or the product in some way. Many of these internal stakeholders might also be users of the product you are interviewing them about.

When To Use It

You can always look for opportunities to interview stakeholders and colleagues to learn more. This is especially useful at the beginning of a project. You can learn expectations for a product, background information on what led to the current status of the project, and goals and hopes for the future. Checking in with stakeholders throughout a project will keep them aware of how things are progressing and allow you to get their feedback. I’ve found this is helpful for building trust with stakeholders and making them feel included in the process.

How You Might Do It

You can often arrange interviews with key stakeholders yourself if they are internal to your company. You’ll identify who is relevant to your project, including project team members, and invite them to an interview. You can contact them to schedule a time, or look to schedule using your company’s shared calendar platform (e.g., Outlook or G Suite). You should know ahead of time how long you need to schedule and how you will interview the participant (in-person or remote) so you can share this information with them at scheduling.

Identifying and interviewing stakeholders becomes more complicated when you don’t have direct access to scheduling yourself. If your project team is part of a larger organization, you might need to ask colleagues in other departments to help identify and schedule stakeholders. If you are on a project team with outside partners or have external stakeholders, you will often need someone to facilitate identification and scheduling of interviews. I’ll cover some additional challenges for recruiting stakeholders and others through your clients in the identifying participants through a client section.

Positive Aspects

Gain insight into roles, backgrounds, and history of stakeholders’ involvement with a product or issue, potentially quick to schedule, low to no cost outside of time, can be done remote or in-person, talking to customer/user-facing stakeholders might provide some insight into what users think of a product.

Negative Aspects

Difficult to identify everyone you want to participate, might include people at high-levels who are hard to reach, scheduling if not doing it yourself, scaling down if resources are limited, does not replace research with users, many stakeholders are too close to their product to be objective.

Incentives

I typically don’t provide an incentive if internal stakeholders are participating during work hours.

Case Study

I worked on a project with a bank that wanted to design an online onboarding experience for new customers. We needed to understand what the current (non-digital) onboarding experience was. We wanted to document available resources to pull into the onboarding experience. Lastly, we needed to build trust with partners who we were going to rely on to champion the experience we created.

We relied on word of mouth to learn who we needed to speak with. First, we interviewed the people we were closest to and asked them who else they considered necessary for us to speak with. We spoke with people in numerous US states, both remotely and in-person. We were able to speak with 30 people in three weeks (this was not a project we were dedicating full time). Occasionally, we spoke to people who were not relevant to our specific purpose. There were two key reasons we were given names of some folks who weren’t relevant:

  • They were higher up executives with little knowledge of what we were exploring
  • The people referring them didn’t understand/effectively convey what we were trying to accomplish, so they volunteered to participate in something not aligned with their role

We found our most common difficulties were in scheduling and getting people to reply to our initial emails. We were trying to schedule an hour to speak with people who spend most of their days traveling and in meetings. Many of them had personal assistants managing their calendars. Some didn’t have an opening to speak with us for weeks after our initial request. Most people did want to make the time to speak with us. They viewed our project as one with high strategic importance in the long-term health of their company. We also had many people reschedule due to unforeseen conflicts involving client needs arising.

We were able to paint a clearer picture of the bank’s onboarding experience and what resources were available. We were able to understand what (some) of the leadership viewed as the potential future for an onboarding experience with new customers and what their perceptions of shortcomings were for the current onboarding experience. We were able to identify gaps in knowledge that required additional future research and education. We made connections with critical internal advocates who walked away with a better understanding and appreciation of the experience we were creating. We would not have been able to achieve these outcomes through a survey or through other means of recruiting participants. Later, we were able to approach these same stakeholders to have them provide feedback on the designs for the onboarding experience we created.

Identifying Participants Through A Client

Many potential research participants are unavailable to the general public. You will find situations where you don’t have direct access to recruiting relevant participants. This is particularly true if you work for a design consultancy/studio, or as part of a shared services team within a large organization. For example, if your client is a widget manufacturer and their product is a widget warehouse product supply application, you will need to access their staff in order to understand their current pain points and needs. You won’t have an easy time finding relevant participants using the population you have access to. You want to conduct research and usability testing with participants who will become the end user of the application, which again means you’d need to access this population through your client.

When To Use It

In addition to the reasons given in the previous section for recruiting stakeholders, when you have to reach specific populations, need opinions from specific people, and want to make your client-stakeholders feel like part of the process. When you don’t have direct insight or access to critical research participants when you are looking to build relationships beyond the project team you are working with, when you want to include a diverse set of individuals covering relevant areas of the product you’re working on.

How To Do It

Work closely with your client or person you are collaborating with to identify the right people for the project you are on. Your project will dictate the exact specifications of roles you need. This includes Product Owners, VPs, Business Analysts, and Users. I often provide a script or email language for my clients to use for recruiting participants. I explain the purpose of the research, how you were made aware of the participant (e.g., Jane from accounting gave me your information) how long the conversation is expected to take, potential dates of availability, incentive (if any), prep work required (if any).

You should provide your client with a screener clearly stating:

  • How many of each type of participant you want to participate
  • Details you want to know ahead of time (e.g., years using the product, industry)
  • Factors leading to disqualification from the study (e.g., less than one year of experience with the product)

Bonus: Many organizations keep data on their users. Your client might be able to screen their database and provide you research participants. However, when I’ve used this in the past, there are often many permissions required and processes to gain access to customers. This can add a significant amount of time to your project.

I am always clear to my clients that scheduling participants is one of the largest hurdles to a project’s timeline. Working with others’ schedules is complicated. You should make it clear to your clients how to recruit, and the need to start recruiting as soon as possible.

Positive Aspects

You get specific people close to a project or product, you learn about long-term and short-term goals directly from the people you work with, you are able to ask to follow up questions that might inform projects well beyond your current relationship, you learn the history of the product or organization, you can reach relevant people you don’t have direct access to, you gain insight into roles, backgrounds, and history of stakeholders’ and users’ involvement with an issue, you will find talking directly to the users of the product provides context and texture you wouldn’t find from someone without similar knowledge.

Negative Aspects

This can be time-consuming, requires a clear communication of purpose, you might end up talking to people less relevant if your client doesn’t screen effectively, less control over scheduling, lack of control over how information is shared with participants.

Incentives – I typically don’t provide an incentive if they are from the client and participating during work hours. I’d provide an incentive if they have recruited users who are coming in on an off day or outside of work hours. You might also have a larger incentive but only give it to a couple randomly selected participants.

Case Study

I worked for a team looking at redesigning a digital report for a large mortgage lender. Many other banks and loan providers do business under the umbrella of this company. We needed to identify a specific type of user, one who: worked for a bank under the parent company and used the report as part of their daily tasks.

The client wanted us to interview 30 individuals with roles interacting with the report. They identified a handful of these individuals upfront, and then put out a call for participation to identify the remaining individuals. There were numerous layers of communication through relationship managers as well as permissions and disclosures the client needed to handle with each participant.

We were able to complete over 30 remote (over the phone) interviews in the month we were allotted to collect data. Our client arranged and scheduled each interview. Our most common difficulties were similar to those I gave in the previous case study, scheduling and relevancy of participants. We were interviewing people who spend their entire workday running the report and using the data to inform their decisions; busy people with limited flexibility of daytime work hours. We made ourselves available at any time a participant had availability in order to solve this. This created drawbacks in scheduling other meetings unrelated to working on the project.

Some of our participants forwarded the invitation to others they thought should be on the interview as well. We would find this out when more than one person would join the call. We were initially caught off guard when we had a call intended for one person take place with four participants at once. We created a separate multi-participant protocol to account for this occurring on future calls, which it did. I recommend expecting this to happen regardless of who is recruiting your participants. It’s difficult to control what happens, once you send out an invitation to the wild.

We used data from our interviews to understand the current behaviors, frustrations, and needs of users. We also presented later participants with sample designs in order to get feedback on report layout and feature changes. We delivered a redesigned report that exceeded client expectations and became a reference piece in their quest to get further funding for research and design projects.

Paying A Recruitment Firm (When You Have An Accessible Population)

Recruitment firms offer services ranging from participant screening and recruitment, facilities to conduct research, recording your sessions, and much more. You can use a recruitment firm when you are conducting research with populations you believe you can reach through contact with the general public. For example, if you are conducting usability testing on an online banking application. You can expect most people familiar with banking transactions (e.g., making a deposit or bill pay) should successfully use your application. Even if they don’t currently use your bank.

I’ve used a number of firms over the past few years. Most of them offer similar services.

Recruitment firms often provide facilities for interviews or usability testing.
Recruitment firms often provide facilities for interviews or usability testing.

When To Use It

When you don’t have direct access to potential participants when you want to have a third party screen your participants, when your sample is available through the general public, when you want to have someone handle recruitment, scheduling, and day-of-research preparation.

How To Do It

You will need to create the screener the recruiter will use. You decide in advance how many of each type of participant you will want. You’ll want to include a number of “floaters” in your recruitment as well. Floaters are people who meet the requirements of the study and are willing to show up for participation in case some of the other participants don’t show up. Floaters are typically compensated at higher levels because they are committing to spend two or three hours sitting around in case they are needed.

You’ll also need to provide the screener with enough advance notice as the recruiter requires. I’ve found this is two weeks in advance for most studies, and three weeks in advance for more complex studies. All recruitment firms offer participants an incentive, usually cash, to participate in a study. You will have to be ok with the fact your participants are receiving money to participate. I haven’t found this to be problematic, but you should be prepared to defend why you don’t think this will add any additional bias to your data.

Positive Aspects

Very detailed screening, don’t have to find people, often have a facility you can use, will record audio and video as needed, will recruit additional participants in case some don’t show up.

Negative Aspects

Cost, the time needed in advance if you have a difficult to reach population, participants trying to game the system.

Incentives – Recruitment firms almost always compensate the people they recruit. You will pay the recruitment firm a set fee they pay to participants.

Case Study

I worked for a team wanting to define the digital needs and behaviors of specific types of Financial Advisors. The client did not want to expose their brand during any of the research, so they did not want to facilitate the recruitment. The client wanted the interviews to pull participants from more than one major city in the US. We worked with a recruitment firm to identify and recruit participants, as well as to conduct the interview sessions.

We worked with the client to create a detailed screener with items meant to refine the population to the specific participants we wanted for the study. The recruitment firm asked for three weeks to find 15 participants for the first city in our study. The usual turn around when working with the firm was two weeks with less specialized participants. We were also advised to provide a higher incentive, over double what we typically offered, due to the probability we were asking participants to step away from work and the perceived value of their time.

We were able to interview 15 participants over the course of two days. We found a few of the participants didn’t actually meet the qualifications we’d screened for. They had manipulated their responses to qualify. Our client was unhappy with this. We were able to use the floaters to replace the participants who didn’t truly qualify. We were also able to get a refund on what we’d paid to recruit the unqualified participants.

Ultimately, we reached our goal of interviewing the right number of participants in the right amount of time, and produced a report on needs and behaviors for our client.

We would not have been able to access this population without the use of the recruitment firm. The client was unwilling to expose their brand and therefore unwilling to identify participants from their contact list. We would have spent more time and money than the project allowed if we were left to recruit participants. We don’t have contact lists or the ability to easily identify specialized populations through our own resources. We still experienced frustration with the lack of initial quality participants the recruitment firm provided. In general, we’ve had positive experiences with recruitment firms, but the more specialized the population, the more likely you will find some duds.

Guerilla Recruiting (When You Want To Find People In The Wild)

You can utilize public spaces to recruit potential study participants. Guerilla research is a term for quick and dirty research conducted with people as they go about their daily tasks (in the wild so to speak). The term is meant to reflect a context in which you are pressed for resources. However, you can benefit from using this method of recruiting even when you have resources for other methods. Sometimes collecting data from people when they are in specific settings is the most appropriate method.

You can find plenty of potential users in the wild.
You can find plenty of potential users in the wild.

You should determine a space you want to recruit participants for a logical reason. Let’s say you’re designing a smartphone application meant to help people track their workouts at the gym. You would want to recruit participants from that setting, entering or exiting the gym. If you wanted to test out a new form of electronic payment, you’d want to be present in a setting where transactions take place.

When To Use It

When you have little time or budget, when you have access to relevant populations, when you only want to get quick feedback from a few people, when you can spend 20 minutes or less per participant, when you have a product related to a specific physical space (e.g., an art museum tour application).

How To Do It

Find a location, get permission if needed, create a script. I’ve previously written a detailed article on the specifics of recruiting participants in public.

Positive Aspects

Quick execution, the potential for multiple locations if you have the resources, small or large sample sizes, accessing relevant populations, compatible with multiple research methods.

Negative Aspects

Little ability for screening, approaching people takes practice and skill, potentially inclement weather if outside, a lot of standing around.

Incentives

I’d base the incentive on the amount of time and type of activity. For example, I might give a product discount code for something taking a minute or less. A $ 5 gift card if you are taking a few minutes of their time.

Case Study

I worked on a project examining the use of technology in library settings. Specifically, we wanted to understand the usability of a system for finding and locating materials within the library. We wanted to work with people who use a library. We needed to test inside of the library because the last part of testing involved physically locating the material.

We sent two researchers to spend multiple days at the library while it was open for patrons. We stood with clipboards at the entrance of the library. We asked patrons if they would spend a few minutes with us participating in our study. We then observed them using the system to search for an item and asked them to locate the material based on where the system told them it should be located.

Our biggest challenge was long periods of time where there were no new patrons coming into the library. We wanted to complete 30 to 40 sessions using three different scenarios. We had budgeted to spend one week onsite to get this many responses. We had to extend our timeline for the following week to reach our goal.

We were able to suggest improvements in the interface, terminology, and an explanation of where materials were located. We would not have had similar findings if we hadn’t been on location at a library and we might not have had as valuable insights if we used people who were not library patrons.

Friends And Family (Low On Time And Budget)

Sometimes, you might have very little opportunity to engage in research. There are many reasons for this, time, budget, or your working for a client who refuses to allow research as part of the project plan. The designers I’ve worked with still want to have some type of feedback to shape their thinking. You can still look to gather some meaningful data from those you have closest access to. Perhaps you are on a project where you are working on a product that is relevant to your coworkers or friends you have easy access to. You might ask a few of them to participate in interviews about the product.

Friends and family are the definition of a convenience sample, and should only be used when no other options exist. This is the most biased and least rigorous way of collecting data. However, you can still benefit from insights into experiences you might otherwise not get. You can use friends and family to participate in interviews or usability testing as a means of accessing informing your design. I strongly recommend conducting additional research, using one of the other methods of finding participants, as your design progresses.

When To Use It

As a last resort, when you have no budget, little time, yet you want to know something about the context or users you are designing for when you have access to relevant people to participate in the study. Background information of your participants.

How To Do It

Reach out to others you and your team know; you can include social media to distribute the call to participate, schedule a time to speak or send an email explaining what you’re asking participants to do (you can also distribute survey links this way) Positive – you will get some feedback, almost instant, low budget

Negative Aspects

Most limited pool of participants, possibly less reach, you’re are relying on favors, less ability to screen for specific characteristics, introducing a larger bias due to familiarity with participants.

Incentives

I would incentivize based on time and budget. A $ 25 gift card is much less expensive than what you’d pay for a participant from a recruitment firm, but friends and family might find this amount acceptable for up to an hour of time.

Case Study

I was part of a project team responding to a (paid) request for proposals (RFP) from a major vacation industry company. We had two weeks to turn around our response, including design concepts to show our thinking. Most of our team had no experience in using the services from this specific industry. We needed to find out more information to help inform our response. We didn’t have the resources to undertake our typical research process of finding and interviewing stakeholders or representative end users. Instead, we reached out to friends and family members who stated they’d had experience in this vacation activity within the past three years.

We emailed our staff and asked if anyone had friends or family members with this qualification who’d be willing to engage in brief phone conversations about their experience. We conducted interviews with seven people over the course of the next two days. Our designers were able to use the insights we gained to better understand the types of needs users might have while vacationing. Our concepts attempted to address some of the issues our participants stated existed when they had experienced while vacationing.

Although we didn’t win the long-term work, our team was able to place among the top candidates. We credited the participation of friends and family in our research as part of what helped our design stand out in a positive way. We were later awarded separate work from the team we presented to for the initial RFP.

The table below provides a summary of key characteristics for each participant recruitment method I’ve covered in this article.

Time Cost Ability to pre-screen participants Ability to access participants
Stakeholder Slow Low Easy Easy
Client Recruits Slow Low Difficult Difficult
Recruitment Firm Slow High Easy Varies – harder to reach specific populations
Guerilla Recruiting Fast Free Difficult Easy
Friends & Family Fast Free Moderate Easy depending on topic

Table 1: Characteristics of common research participant recruitment methods

Conclusion

We need to access users and potential users in order to effectively conduct research. I’ve covered a number of common ways you can find research participants. Each has certain strengths and weaknesses. You’ll want to become familiar with each of these and adapt your approach based on your product, budget, and timeline.

Smashing Editorial (cc, ra, il)


Articles on Smashing Magazine — For Web Designers And Developers

17 Jan

A Comprehensive Guide To UX Research

by TBSCategories: News

(This is a sponsored article.) Before embarking upon the design phase of any project, it’s critical to undertake some research so that the decisions you make are undertaken from an informed position. In this third article of my series for Adobe XD, I’ll be focusing on the importance of undertaking user research. Your job title might not be “design researcher”, but that doesn’t mean you shouldn’t at the very least inform yourself of your users and their needs by undertaking at least some initial scoping research before you embark upon a project.
Articles on Smashing Magazine — For Web Designers And Developers

11 Dec

Dealing With Stress As A Designer: Here’s What Research Says

by TBSCategories: News

The world is a stressful place. That’s probably why if you search for articles about dealing with stress you’ll find lots of great advice. But the problem is that most of it is pretty generic. Eat good food. Exercise. Get plenty of sleep. Those are all good general tips — but is there anything more specific to the profession of design? If you’re a designer who feels stressed from time to time, then this article is for you.
Articles on Smashing Magazine — For Web Designers And Developers

advertisement-24
30 Mar

Filling Up Your Tank, Or How To Justify User Research Sample Size And Data

by TBSCategories: News

   

Jen is presenting her research report to a client, who runs an e-commerce website. She conducted interviews with 12 potential users. Her goal was to understand the conditions under which users choose to shop online versus in store.

Filling Up Your Tank, Or How To Justify User Research Sample Size And Data

The client asks Jen why they should trust her research when she has spoken to only 12 people. Jen explains her process to the client. She shares how she determined the sample size and collected and analyzed her data through the lens of data saturation. The client feels comfortable with the explanation. She asks Jen to continue the presentation.

The post Filling Up Your Tank, Or How To Justify User Research Sample Size And Data appeared first on Smashing Magazine.

Smashing Magazine

Newer postsOlder posts
×

Powered by WhatsApp Chat

WhatsApp Chat is free, download and try it now here!

× Chat with Us