Currently, the Vector 2022 skin is the second most-popular non-default skin on English Wikipedia. It is also in use as the default across over 20 community-led projects accounting for more than 1 billion pageviews per month, such as French Wikipedia, where it has been the default since 2020.

Current usage of non-default skins on English Wikipedia as of August 30

The objective for the Vector 2022 skin is to make the interface more welcoming and comfortable for readers and useful for advanced users on desktop. To do this, the skin introduces changes to the navigation and layout of the site, adds persistent elements such as a sticky header and Table of Contents to make frequently used actions easier to access, and makes some changes to the overall styling of the page. The analysis of the data collected concluded that these changes improve readability and usability, and save time currently spent in scrolling, searching, and navigating – all of which can be interpreted to create an easier reading experience. The new skin does not remove any functionality currently available on the Vector skin.

Below, you'll find more details on how the skin was built over the past three years, what has been changed, and why these changes are considered to be improvements based on the completed research and data collected.

About[edit]

The slides for our presentation at Wikimania 2022. You may also listen to the recording on YouTube.

[Why is a change necessary] The current skin, Vector, has been in use since 2010. When it was developed, it reflected the needs of the readers and editors of the Wikimedia sites in that year. (See the Wikimedia Usability Initiative wiki for more information.) Since then, vast new audiences have begun using the Internet and Wikimedia projects. Research done with these audiences showed that the current default skin doesn't meet their needs. The Vector 2022 skin aims to change the interface in ways which include the needs of all of the current audiences – both those who have been using the projects for a long time, as well as those who have joined more recently, or have yet to join.

[Improvement in usability] The objective for the skin is to make the interface more welcoming and comfortable for readers and useful for advanced users on desktop. It builds upon the existing Vector skin to introduce a series of feature changes that aim to improve problems new and existing readers and editors were having with the old skin. It also draws inspiration from previous user requests, the Community Wishlist Surveys, and existing gadgets and scripts across different projects.

[Technical improvement] The renewed product focus allowed to bring Vector in line with the development standards and improve all legacy skins. PHP code in Wikimedia deployed skins has been reduced by 75%. The project has also focused on improving gadget support across all skins by making sure skins generate consistent HTML and have improved APIs (for example handling icons in menus).

[Try it out]

[Effectiveness] The results and analysis of A/B testing and qualitative testing concluded that these changes make it easier to read and learn, navigate within the page, search, switch between languages, use page and user tools, and more, without negative effects to pageviews, account creation, or edit rates. The team has been working on the new skin for the past three years, ensuring that every change is tested and proven to work. For more details, look at the Findings and results.

Process for developing the new skin[edit]

The new Vector 2022 skin has been in development for the past three years, and tested in collaboration with different audiences and communities on English Wikipedia and other wikis.

  1. Problem identification research with both readers and editors – During this phase, in 2019, the Web team studied the way people used the site and identified the largest usability issues as well as issues to exploring the site further, becoming more engaged with reading or editing. The team did this by interviewing readers and editors across multiple countries and locations. They also surveyed existing gadgets and use scripts across different wikis and communities to identify what types of changes and alterations to Vector legacy (2010) were made to improve readability and usability (See the links: Research and design: Phase 1, Research and design: Phase 2.)
  2. Prototype development and testing – Over the course of 2019–2022, the team built out the ideas for individual features and began showing potential solutions to our audiences. These ideas were built in the forms of usable prototypes. Each feature was tested with readers and editors through interviews and wider rounds of prototype testing. Generally, for testing with editors the team used Central Notice banners across multiple languages and Wikimedia projects, including English Wikipedia, so that the widest audience possible could be involved. Each prototype was tested by approximately 200 editors on average. (Example)
  3. Refining and building – During 2020–2022, the feedback from the prototype testing was taken, and the prototype was refined or changed based on what needs were identified in the prototype testing from both readers and editors. In some cases, the team asked for additional feedback once changes and iterations to ensure they were making the right decisions.
  4. A/B testing and other quantitative testing on pilot (early adopter) wikis – For each feature, the team performed quantitative tests for whether the features worked as expected based on the previously defined criteria of success. For example, the sticky header was designed to decrease scrolling to the top of the page. The team gave the sticky header to 50% of users and compared them to the other 50% for two weeks. After two weeks the results were compared and it turned out that people who had the sticky header were indeed scrolling less to the top of the page in order to select any of the tools available there. If the team received negative results from the test, they focused on identifying the reasons for these results across different audiences and made changes to the feature which addressed them. Then, they would test the feature again. This was the "beta" phase. During this phase, the team also monitored usage across all wikis, including English Wikipedia, where many account holders have been already using the new skin.
  5. Finally, now, the team would like to deploy Vector (2022) on more wikis and continue monitoring the way people are using it so that they can flag any issues.

New features[edit]

The skin includes changes to the layout of the site, location and prominence of some features, the overall readability, and addition of sticky features. Among the best-received by the communities, there are the new Table of Contents, sticky header, and the search widget. No existing features or tools were removed as a result of the new skin.

Findings and results[edit]

The results of testing and subsequent analysis concluded that these changes result in improved usage of the skin, and save time currently spent in scrolling, searching, and navigating – all of which can be interpreted to create an easier and more comfortable reading experience. There are no negative effects to pageviews, account creation, or edit rates. Please visit the repository page for the full versions of research and reports.

Research Usability Usage Sentiment Health Metrics and Overall Performance
Why This helped identify what needed to be fixed This helped verify that the suggested solutions were correct This helped verify that the solutions performed better than their previous iterations This helped estimate the types of opinions would be heard about the skin immediately after deployment This helped estimate the long-term effects and benefits of using the new skin at scale
What Research across multiple languages and wikis, including English Wikipedia, was performed to determine the main issues with the old Vector skin User testing, prototype testing, and interviews were performed for individual features as well as the overall skin A/B testing and subsequent monitoring was performed to validate the results of qualitative testing on wikis, to show how features and the skin were used in reality Surveys were taken to estimate the subjective reactions to the new skin at the moment of deployment In addition to testing individual features, health metrics were identified and monitored to ensure the usage of the new skin was on-par or better than the old Vector skin
Type Qualitative Qualitative Quantitative Survey Qualitative
Results
  • Too much information density and navigational elements on the page made it difficult to focus on a particular reading or editing task.
  • Readers were unable to understand the purpose, terminology, and concepts of available tools such as the main menu, user menu, and page tools.
  • Readers found it difficult to navigate the page and spent a lot of time scrolling
  • Readers found it difficult to search for information that is relevant to them.
  • Multilingual readers were not aware of language switching functionality.
  • Readers wanted to have an easy way to view the table of contents throughout the page.
  • Editors and other contributors wanted easier access to tools they used frequently.
  • Editors found the hierarchy of the navigation to be confusing for newcomers.
  • Editors expressed interest in having fewer, but more prominent, ways to begin contributing.
  • Editors expressed the need to be able to configure the interface via gadgets and user scripts.
  • Research into the width of the content indicated that Wikipedia's character count per line was more than three times the recommended value.
Header Long-term affinity surveys Opt-in rates
In prototype testing with editors, the large majority of users reported positive to neutral experiences with the new header and logo configuration. 8% of editors tested reported negative experiences with the new logo. There was a survey on the general sentiment towards the site in its current (August 2022) form. A few months after deployment, this survey will be run again to identify the ways people will feel towards the new experience. On the initial partner communities, 96% of all logged-in users and 88% of active editors decided to keep the changes once the default was changed (the target was 60%).
Collapsible sidebar First look sentiment surveys Usage
  • In prototype testing with editors, the majority of users who provided feedback liked the collapsed sidebar for personal use, and especially for the purposes of reading.
  • In a user study with readers, participants reported a strong preference towards the collapsed version of the menu, stating that it created a better reading experience than the current version.
  • Logged-in and logged-out users have very different behavior while using the sidebar.
  • Both logged-out and logged-in users used the collapsible sidebar frequently.
  • Logged-out users tended to collapse the sidebar and leave it collapsed 81% of the time.
  • However, the rate of uncollapsing indicated that users are aware that, were they to need to navigate to an item in the sidebar, that option was available to them.
  • Logged-in users tended to collapse and then uncollapse the sidebar at fairly even rates, indicating that they were collapsing it when necessary and uncollapsing to use the necessary functionality.
There was a survey identifying the sentiment of people upon their first interaction with the new skin.
  • Overall, the majority of logged-out users reported that they would view the changes either positively or neutrally.
  • A significant number of resposes indicated preference for the current skin and resistance to change.

These results are expected and on-par with similar design changes made by other websites. They will be used to prepare communities and internal communications for some amount of negative feedback upon deployment.

  • Currently, the Vector 2022 skin is the second most-used non-default skin on English Wikipedia, with 9301 opt-ins.
  • The most-used non-default skin was monobook with 13494 opt-ins.
  • The third most-used non-default skin was timeless with 2435 opt-ins.
Limited content width Pageviews
  • In prototype testing with editors, most editors appreciated the shorter line lengths and agreed that the feature created a more comfortable reading experience.
  • A significant number of editors disliked the whitespace around the content and felt that it was wasted space.
  • In user testing with readers, participants reported a strong preference to the limited content width, stating that it improved the reading experience.
  • Previous research indicated that users read more accurately and more quickly at limited line widths.
Limiting the content width did not show any significant effects of the health metrics: pageviews, edit count, and account creation across the pilot wikis.
  • Wikis with the Vector 2022 skin as default did not experience any decreases in pageviews attributed to the Vector 2022 skin.
  • Note: without direct A/B tests across all users, which is not privacy-friendly, increases in pageviews due to the new experience are difficult to prove.
  • This is why a threshold of no significant increase or decrease is used instead.
  • However, wikis with the Vector 2022 skin as the default are currently receiving a higher ratio of pageviews when compared to the 2019 benchmark than English Wikipedia or all other Wikipedias with the exclusion of English.
Search Edit rates
In user testing performed on usertesting.com, all participants were able to find the result they were looking for using the new search.
  • There was an average 28.9% increase in the search sessions initiated across early adopter wikis in the AB test for the group using the new search widget compared to the previous search widget for logged-in users.
  • For logged-out users, there was a 13.4% increase in search sessions initiated following the deployment of the new search widget.

Wikis with the Vector 2022 skin as default did not experience any decreases in edit rates attributed to the Vector 2022 skin.

Language switching Account creation
  • In prototype testing with editors, the majority of editors preferred the new location of the language selector.
  • However, a significant number of editors expressed concern with having an extra click in order to switch languages.
  • In user testing with readers, participants reading in non-English languages, as well as multilingual participants reading in Englsh were able to find the new language selector more easily and to switch to a different language quicker.
  • In an A/B test for the language switching functionality with logged-in users, there was a 44.57% decrease in total clicks on language links. This led the Web team to iterate on the feature.
  • After iterating on the feature to include guidance on where the new functionality is located, there was an increase in language switching for logged-out users using the new version of the language switcher and comparable usage of the new and old versions of the language switcher for logged-in users.
  • Wikis with the Vector 2022 skin experienced minor increases in account creation attributed to the Vector 2022 skin.
  • A thorough A/B test will have to be performed to establish whether these changes are significant.
Sticky header
  • In user testing with readers and editors, two issues were studied: the ease of use of a sticky header, and the highest-priority functionality to include within the sticky header.
  • Based on these results, the icons within the current sticky header were chosen.
  • In prototype testing with logged-in users, an overwhelming majority of test participants reported positive experiences with the sticky header.
  • There was an average 15% decrease in scrolls per session by logged-in users on the 15 pilot wikis in the treatment group (with the new sticky header), compared to the control group (without the sticky header).
  • This indicates that the hypothesis was correct - adding the sticky header to the page reduced the need to scroll to the top of the page significantly.
User menu
  • In prototype testing with editors showed that the new user menu was easy to use and participants could intuitively navigate to pages within the menu from their first try.
  • Some participants indicated they would like more control over which items are shown within and outside the menu.
  • User testing with readers and editors indicated that the menu items and icons were intuitive to use and understand.
Table of contents
  • User testing with readers in multiple countries revealed that readers and editors prefer a table of contents that is persistent throughout the page, without any collapsible elements.
  • In prototype testing with editors, the overwhelming majority of the feedback received on the table of contents was positive. Logged-in users also reported potential changes to the proposed ToC that would improve its usability.
  • Among the sessions with at least 1 click on ToC, the treatment group (the group that is exposed to the new ToC) has more clicks on ToC than the control group (the group exposed to the old ToC).
  • The data model predicts 53% more clicks on new ToC with logged-in users and 45.5% more clicks on new ToC with anonymous users when compared to the previous ToC.
  • This trend is consistent across all edit count buckets: i.e. logged-in users clicked on the ToC at roughly the same frequency regardless of how many edits they had previously made.
References Features

Sentiment Survey

Tasks which will be completed before the deployment[edit]

  1. [DONE] Table of Contents collapsing and narrow screens behavior – This work is now completed. The table of contents is collapsible, and can be accessed from both the title of the page as well as from the sticky header.
  2. [DONE] Visual refinements – The team is currently wrapping up the core parts of the visual refinements work. Please see the Qualitative Testing section on the project page dedicated to this part of the project for a full list of the changes Web plans on making.
  3. [In Progress] Coordinates – Web is continuing to explore different solutions for coordinate alignment, including potentially adding coordinates directly into the styles of the skin in the future.

Support for Vector 2022 after deployment[edit]

We plan to continue work on the desktop experience after this change is made, and into the future. Here's a snapshot of the projects we will be working on within the next six months.

  1. October–November 2022: Once the skin is deployed, we will be focusing on the page tools feature, which will allow easier access to page tools and further customization of our menus and tools overall. We encourage you to check out the prototype for this work.
  2. March 2023: Navigation improvements – we are currently exploring ways to make it easier and clearer to understand how wiks and the content creation process works for readers, and specifically surfacing the fact that wikis are collaborative spaces anyone can get involved in. Our goal is to enable people to read content more critically and be able to recognize what is trustworthy, as well as to make it easier for those who might be interested to know that they can contribute and to begin their contributions journey. We will be collaborating on this work with the Growth and Editing teams (see: Core Experiences).