This is an archive of past discussions. Do not edit the contents of this page. If you wish to start a new discussion or revive an old one, please do so on the current talk page.
Right now the toolserver is having problems, and the data there that the bot uses is several weeks out of date. I can debug the problems with your project, but until the toolserver is fixed even if your project is set up correctly it will not work until the toolserver catches up with the data for your project. I will keep an eye on it and see what happens. I'm sorry to say I cannot fix it may take a couple weeks before I can get it working, because of the toolserver delay that is out of my control. — Carl (CBM · talk) 02:21, 5 August 2012 (UTC)
Date/Time Stamp
Given that the bot can't keep the tables updated in real time, would it be possible for it to add run-time Date/Time stamps so that viewers can see how old the information is? (Judging by the assessment log and my own edits, the WP:TREK tables are somewhere between one and a half and two and a half weeks old right now...) Cbbkr (talk) 22:48, 4 August 2012 (UTC)
Yes, this is not too hard to do. But the problem is that the tables are often out of date not because of the bot, but because the data on the toolserver is out of date. The bot thinks that it is updating the tables every day or so. In general, if there is no change in the table data, the bot will not re-upload the table. Now if there was a timestamp, it would be clear that the bot has regenerated the table, but the data in the table would still be out of date. So if the bot put a timestamp on the table, I am afraid it might mislead people to think that the data is up to date. — Carl (CBM · talk) 02:18, 5 August 2012 (UTC)
Thanks... The implication appears to be that there is no bot-accessible audit trail (or something similar) that can tell it when the toolserver's data was last updated (That couldn't be addressed, could it?). Cbbkr (talk) 22:09, 6 August 2012 (UTC)
There are graphs of the replication lag online at [1]. The problem is that, to work out what data a table would have, you would have to know the replag at the time the table was created. The general rule is that the delay in data getting into the table is about (toolserver replag + 1 day). — Carl (CBM · talk) 03:16, 7 August 2012 (UTC)
WP 1.0 bot temporarily down 2012-8-7
The bot is temporarily down pending two issues. The first is the toolserver replag, which is excessively high, preventing the bot from seeing data up to two weeks old. The second, which may be related, is a technical issue with database tables appearing to be locked while the bot is running. Until further notice, the bot will not be making on-wiki updates and the web tools have been disabled. I apologize for the delay, but there is simply no way to make the bot run in a functional way until these issues are resolved.
If other bot developers are interested in assisting with these problems, I would be very happy to add you to the WP 1.0 multi maintainer project on the toolserver. — Carl (CBM · talk) 12:46, 7 August 2012 (UTC)
Further issue: It appears that the toolserver has been reorganized completely. As a result, all of the numerical links in the data tables return a "Forbidden" message instead of the usual listing of articles. Possibly not related, but it is another problem to be addressed if the process is to continue working. --EncycloPetey (talk) 22:13, 8 August 2012 (UTC)
The "Forbidden" is expected. I hope that over the weekend I can code a more gentle way of turning off the web tools, but for now I have just forcibly blocked everyone from accessing them. On the plus side, it appears I may be able to do some limited updates, although the data will still lag behind because of the toolserver replag. But the web tools will be unavailable for a while, although I hope to make that as short a time as possible. — Carl (CBM · talk) 01:48, 9 August 2012 (UTC)
Partial progress 2012-8-10
Running the bot manually, I was able to do an update of all the non-Biography projects and upload project tables, which took the last two days. I am running another update, after which I will upload tables and logs if everything goes smoothly. — Carl (CBM · talk) 18:13, 10 August 2012 (UTC)
Here is a status update. The toolserver has two database servers that hold copies of English Wikipedia, one of which also has the custom data for WP 1.0. All the custom data is being copied to the other database server, which has lower replag. Once that is finished, I should be able to run another update of all the on-wiki tables and logs. The database server that WP 1.0 is currently using (where the data is being copied from) will go offline for maintenance after the data is copied from it. Because toolserver will be down to just one enwiki database server temporarily, the web tool will remain offline to help limit the load on that server. Once both enwiki database servers are back in operation, I should be able to re-enable the web tool. — Carl (CBM · talk) 11:35, 15 August 2012 (UTC)
Update: 2012-8-25
I am trying a full update on the toolserver today. They have moved the WP 1.0 database and all other "user" databases to the other enwiki database server on toolserver, which should have less replag. If the update is successful, I will then test re-enabling the web tools. A typical update takes only about 1 day, but when there has been a long pause like now it will take several days to do the update and upload all the tables and logs (part of the extra delay is due to the page moves, which take extra work to process; the other part is due to extra time to upload the assessment logs to the wiki). — Carl (CBM · talk) 14:13, 25 August 2012 (UTC)
The update of the WP 1.0 database is done, and the updated project summary tables are uploaded to the wiki. Started the log upload now, it will take some time. — Carl (CBM · talk) 12:28, 26 August 2012 (UTC)
You're welcome. The log uploads took about 24 hours, and I also uploaded a new global statistics table. I should be able to get the updates back to every day or two now. I will work on the web tools next, and then the backlog of other issues on this page. — Carl (CBM · talk) 22:00, 27 August 2012 (UTC)
I only noticed that this is down today. I hacked a quick interim tool for rudimentary display (demo) that uses only the Wikipedia API, not the toolserver databases. Can be improved if there is interest. --Magnus Manske (talk) 21:46, 29 August 2012 (UTC)
I have re-enabled the web tools except for the "update project data" tool. The other ones seem to be a little slow, but I am not sure why. The "update" tool has always been problematic in terms of performance. The backend for it needs to be rewritten, and until I do that I am going to leave it disabled. If anyone else is interested in helping with the coding, I am always looking for additional maintainers. Of course the project data will still be updated in the background, just not "on demand" (not that the "on demand" worked very well). — Carl (CBM · talk) 21:51, 2 September 2012 (UTC)
Good to see the overviews being updated again, even if the 'update project data' is still disabled. Did an update action on WikiProject Tennis, could you run an update of that overview? Thx.--Wolbo (talk) 10:52, 6 September 2012 (UTC)
I will be glad to help, but until the toolserver issues are resolved the bot is not running. I will check the set up of your project the next time that I run the bot. I'm sorry for the delay, the toolserver issues are unfortunately out of my control. — Carl (CBM · talk) 12:43, 18 August 2012 (UTC)
Articles by quality logs haven't been updated in a while - since August 26, apparently. Is everything OK or there is still a technical problem with the bot runs? GregorB (talk) 11:52, 20 September 2012 (UTC)
I had turned off the log updates because of the high replag. I turned them back on now, and started a job to upload them. If things go well all the past logs will be uploaded to the wiki in the next 24 hours. The web tools always have the updated logs based on what data has been collected from the wiki - these are at http://toolserver.org/~enwp10 . — Carl (CBM · talk) 02:19, 27 September 2012 (UTC)
I'm trying to help make the WikiProject Graffiti look more loved and would be delighted if an article quality stats table could be generated for the tagged articles. Any chance of that? Thanks in advance. Sionk (talk) 10:27, 11 September 2012 (UTC)
When the target of a redirect is changed, why is the redirect itself listed as "Renamed" when no files are actually changing names? It seems to me that this should be a separate section apart from true file renamings. Cbbkr (talk) 18:08, 28 September 2012 (UTC)
"Renamed" means that there was a page A that used to be assessed, but now it is not. When the bot sees this, it then checks whether there is a redirect from A to some other page B. If there is, it will say in the logs that A was renamed to B instead of saying that A is now unassessed and B is now assessed. The bot does not go on to see what else might have happened with the redirects, it just notices that page A is no longer in the assessed articles list and A is a redirect to another page. — Carl (CBM · talk) 20:02, 28 September 2012 (UTC)
The online "update now" tool will be disabled for a while - it was not working properly and parts of it have to be rewritten. But the bot does update each project on a daily or near-daily basis automatically. — Carl (CBM · talk) 12:05, 9 October 2012 (UTC)
My mistake, I thought the 'bot' referred to the 'update' function. FYI the WP:TENNIS overview has been updated a few times recently but the last time was about two weeks ago.--Wolbo (talk) 13:20, 10 October 2012 (UTC)
Thank you for letting me know - the bot has been collecting the data, but a separate toolserver issue turns out to have been killing the upload process. I made some tweaks today that should help it, and I will watch it for a few days. The new data is uploaded now [2]. The toolserver is surprisingly flaky, so I depend on error reports to know when a new problem has come up. — Carl (CBM · talk) 19:01, 10 October 2012 (UTC)
Not sure if there is an issue but WP:TENNIS has not updated for about four days when previously it updated daily or at least every two days.--Wolbo (talk) 07:45, 15 November 2012 (UTC)
Yes. The toolserver admins are re-installing the enwiki database replica this week to get rid of the corruption mentioned in a lower section of this talk page. I left the WP 1.0 web tools up, although they may break from time to time while the databases are re-installed. But I turned off the automatic updates until the database maintenance is done, which should be later this week. — Carl (CBM · talk) 11:21, 15 November 2012 (UTC)
Sortable columns on toolserver?
Would it be possible to make the project and article columns sortable when all WikiProjects are listed on toolserver (i.e. here). As a writer for the Signpost's WikiProject Report, I'd like to easily see which projects are the largest and be able to determine which have been growing most rapidly each month. -Mabeenot (talk) 05:17, 3 October 2012 (UTC)
It is hard to sort that page because the list is broken into separate tables for each letter. Instead I just hacked a quick and dirty report that will show you a sorted list. It is at [3]. However, I know that the counts for all the Biography projects are wrong. Because of toolserver performance problems those are rarely updated. Most of the other projects should be up to date. I will work on making some tweaks to the code to see if I can get the Biography projects to update more regularly. — Carl (CBM · talk) 11:13, 3 October 2012 (UTC)
The report I sent you is dynamic: it always uses the latest data, so it will update itself. I suppose we could set up something to copy it to the wiki every month if that would help. — Carl (CBM · talk) 10:40, 4 October 2012 (UTC)
Everything seems fine to me. You tagged an article as B-class on 6 October, then the bot generated the table on 10 October with the single entry. Since then you have tagged two more articles on 11 October. The two new articles should be added to the table the next time the bot cycles through your data. If the table still hasn't updated in a week I will be happy to look at the situation again. Road Wizard (talk) 21:03, 12 October 2012 (UTC)
Thank you for taking a look. I did tag that article for importance too. Is there a reason the table doesn't show that scale too? And as I tag more articles will the table adjust automatically to look more like the standard table used on WikiProject pages? Sorry for all the questions. I'm not the most tech savvy person in the world :) -AngelKelley (talk) 21:16, 12 October 2012 (UTC)
Sorry, I missed your point about importance. I think it is isn't identifying separate columns yet because you just have 1 high importance article. When you have more articles with other quality or importance ratings the table should expand. Don't worry about asking questions, these templates and tables are confusing to everyone in the beginning. Feel free to ask anything else if you have any other issues. Road Wizard (talk) 21:32, 12 October 2012 (UTC)
The table already looks more like a "regular" table. The table only shows the rating levels that it actually noticed the previous time it gathered data from the project. There is often a lag of a day or two, because after you save a change the bot has to gather data from your project (this runs once a day) and then upload the table back to the wiki (this usually happens the next day). Recently, the "update" didn't run one day because of an issue when the toolerver admins changed their job scheduling system; this added another day of delay. But if the categories are correct on the article then the bot will indeed gather the data the next time it runs. — Carl (CBM · talk) 21:39, 12 October 2012 (UTC)
The bot stopped updating the first page almost a year ago so there shouldn't be any problem with deleting it from the bot's perspective. However you will lose your project's historic record of article improvements. Whether your project wants to retain that historic record is another question... Road Wizard (talk) 05:22, 15 October 2012 (UTC)
Please don't "clean up" any pages in the bot's user space. These are in user space to emphasize that the bot can edit them at any time, even if it seems like they are out of date. The other reason they are in the bot's user space instead of Wikipedia space is to emphasize that they are the responsibility of the maintainer. One reason these are kept is that there is no other record of the past article counts that the bot has generated, and these counts might be of interest to data gatherers, not just to the wikiproject itself. The job of the bot is not just to make current tables - it is to keep a permanent record of old assessments via logs and page histories. — Carl (CBM · talk) 10:54, 15 October 2012 (UTC)
Article size
Would it be possible for the tool to display each article's size? This is something that I'm interested in (i.e., comparing the size of articles in the same rating-class). Thanks! Toccata quarta (talk) 22:13, 24 October 2012 (UTC)
Here is an initial attempt [4]. It still has a few bugs but it should be in a state where you can try it and let me know if it lets you make the comparison you want. — Carl (CBM · talk) 17:48, 25 October 2012 (UTC)
WP 1.0 maintainer stepping down
To give myself more time to work on other interests, I am going to step down as the WP 1.0 bot maintainer at the end of November 2012. The WP 1.0 bot is used by over 1,000 wikiprojects and also used to make released offline copies of Wikipedia. I am looking for a new maintainer who is interested in taking over this important job. I want to announce this well in advance to allow for a smooth transition. I am not retiring from Wikipedia as a whole; I am just turning over the bot to someone else.
The new maintainer needs to have some programming skills and experience with Linux/UNIX. But the details of the bot can be learned "on the job"; you don't have to be an expert, and this project would be an ideal way to gain some real experience with a nontrivial bot and web programming project. If you are interested, please let me know or comment here. You are welcome to email me if you would like to discuss things privately.
Here are a few important details:
The WP 1.0 bot runs in a "multi-maintainer project" on the Wikimedia Toolserver. It is straightforward to get a toolserver account and for the toolserver administrators to add another toolserver account to the project.
I will be available for consultation during the transition to a new maintainer, and even after someone else takes over. However, I am not looking for a co-maintainer - I am stepping down and a new maintainer is needed.
There are many interesting things going on, such as the possibility of transitioning the bot to Wikimedia Labs after that becomes fully operational. The new maintainer would be able to help shape the course of the WP 1.0 project.
This sounds like an interesting project; I'd be interested in helping out/maintaining it (and am adequately knowledgeable in UNIX-esque fields...although PHP is my language of choice ) -- if you'd like to send me an email to further discuss, I'd be happy to talk! TheopolismeBoo!17:23, 28 October 2012 (UTC)
I'm interested in this too. If Theopolisme doesn't take this on, I'd be happy to. I already have a Toolserver account and experience with Linux & programming. — Wolfgang42 (talk) 01:11, 22 November 2012 (UTC)
You cannot see it on the wiki, but there is some corruption in the toolserver database, which caused this article to look like it was in multiple assessment categories when the bot consults the toolserver database. On an article by article basis, when problems are noticed, I can fix it by deleting the talk page and restoring it, which cleans up the list of categories in the toolserver database. The global solution is for the toolserver admins to re-import a dump of the enwiki database. They are in the process of working with the WMF admins to get a fresh dump to do that. — Carl (CBM · talk) 13:38, 31 October 2012 (UTC)
Yes, it's the same problem. I deleted the page and undeleted it to cause the toolserver database to refresh the categories. There have been some emails on the toolserver-l mailing list about progress on getting a new dump; it will be a few days at least. — Carl (CBM · talk) 20:24, 5 November 2012 (UTC)
Hi everyone. The table generated by WP 1.0 bot for applied linguistics isn't working how I expect it to, and I was wondering if anyone here could help. The Applied Linguistics Task Force used to have task-force-specific importance ratings, but it doesn't any more. The problem is that some of the importance categories still appear in the table when they shouldn't (even after the categories themselves have been deleted). Does anyone know what's going on here, and whether it can be fixed? Best regards — Mr. Stradivarius(have a chat)15:23, 21 November 2012 (UTC)
Hi Mr. Stradivarius. As with this and others, the problem seems to be Toolserver database corruption (of the actual Wikipedia database) combined with a rather confused bot — articles appearing in 'phantom classes' and such. Carl's (user:CBM) been deleting/undeleting pages, which seems to help a bit. Please hold for the next available representative... —Theopolisme03:58, 22 November 2012 (UTC)
Thanks for the reply, it is much appreciated. If deleting and restoring the talk pages will fix this particular case, then I can do that when I get back on my admin account as there are only two individual pages involved. For anything more than that I'm happy to wait for the new database dump. — Mr. Stradivarius on tour(have a chat)04:57, 22 November 2012 (UTC)
The original problem was the database corruption. But even once you deleted and recreated these, the bot would not have fixed the problem, because there were no importance categories at all any more for this project. The bot was therefore never going to update the importance ratings; this is a safety measure in case someone accidentally deletes the 'articles by importance' category for the project.
So I fixed the problem manually for the two articles in question, and then had the bot recreate and upload the table. You can see on the new version that, once all the importance ratings are gone, the bo no longer tries to sort the articles by importance.
For co-maintainers: when I was fixing this I put a log into typescript.linguistics.20121122.txt in the top level directory of the enwp10 project on toolserver. So you can see in there the commands that I used, which were some SQL commands and then an manual update of the project using the bot. — Carl (CBM · talk) 14:53, 22 November 2012 (UTC)
Thanks for looking into this for me! I really appreciate it, and I want to thank you for everything you've done with the WP 1.0 assessments. I'm just getting a sense of the amount of work that has gone into this, but even with my limited understanding I can tell that it's A Whole Lot. Thank you. :) — Mr. Stradivarius(have a chat)15:17, 22 November 2012 (UTC)
Updates
What is the current situation regarding the frequency of updates? It was mentioned recently that the assessment overviews should update every day or every other day but this seems to be hit and miss. For instance the WP:TENNIS overview has not been updated in at least the last five days. It would be good if we can get some stability and reliability in this, at least until the option for the users to run their own updates has been re-installed. Thx. --Wolbo (talk) 07:59, 30 November 2012 (UTC)
They are infrequent right now because the toolserver admins have been planning some maintenance on the databases, and so I am running the updates manually instead of on a fixed schedule. Once that database maintenance is done, things will go back to the regular schedule. An email to the toolserver-l list this morning says they have finished one of the main steps of the maintenance, so things are moving forward. I'm sorry for the inconvenience, — Carl (CBM · talk) 12:33, 30 November 2012 (UTC)
It appears I misread the email. The latest news on toolserver-l is that they are going to request a fresh dump from the Wikimedia people, and then they will be able to do the maintenance. The problem is that there is some bad data in the toolserver database, which can be fixed by basically reinstalling it from a fresh copy (dump) of the live enwiki database. But that copy has to be very recent in order for the history files to still be available to catch up from the copied version to the latest state of the database, after which point the two are kept in sync. Those history files are very large so they are not available very far back; if the dump gets too stale then it has to be regenerated by Wikimedia before the toolserver people can use it to reinstall the database. I will post another update here when I hear anything new. — Carl (CBM · talk) 22:00, 3 December 2012 (UTC)
Category search enhancemet
This must surely have been asked for before but it would be very useful if category search were enhanced to have a recursive option (so it could go into subcategories).
Category loops you could doubtless catch.
To avoid the problem of potentially very deep recursion, perhaps a user specified depth parameter would work; a value of four should suffice for the majority of cases.
This could be done on a case by case basis. However, it is relatively inefficient, so it would be hard to have the web tool do it, because it would take too long and the web request would be cut off by the webserver. If it was done by hand from the command line on toolserver, it would have more time to execute. Is there a specific category that you need? — Carl (CBM · talk) 13:47, 11 December 2012 (UTC)
Not that would be worth putting anyone to that trouble for just now but thanks for the offer; might be useful in future. -Arb. (talk) 18:34, 11 December 2012 (UTC)
New maintainers for the WP 1.0 bot: Theopolisme and Wolfgang42
I announced one month ago that I am stepping down as the WP 1.0 bot maintainer [5]. I am very happy to announce that two people have contacted me about taking over the enwp10 project: User:Theopolisme and User:Wolfgang42. They have now been given the technical rights on toolserver that will allow them take over the bot maintenance. I want to wish them the best with the project, which has been a lot of fun for me to work on the past few years. I will still be available on the wiki; I am stepping down from the WP 1.0 bot to turn my efforts to some new projects. — Carl (CBM · talk) 13:42, 11 December 2012 (UTC)
I was interested in trying out the script and seeing if it actually did work--if not, I figured we could unlink it/etc. —Theopolisme13:55, 21 December 2012 (UTC)
I've made the edit. Now I see that only a piece of the page is protected, and it has been edited more recently [6]. — Carl (CBM · talk) 14:02, 21 December 2012 (UTC)
This script dates back to the maintainer before me. I don't know whether it predates or postdates some of the significant changes Mediawiki that required reimplementing the code in bots to log in and edit the wiki. If it predates that, then it will certainly not work. — Carl (CBM · talk) 14:04, 21 December 2012 (UTC)
Well, it crashed Safari the first time I tried to run it. Trying again now. If it still doesn't work, I'll take a look at the code; we might be able to salvage it. —Theopolisme14:36, 21 December 2012 (UTC)
However, doesn't look like those were actually created...so yep, I think you're right (as far as the login mechanism goes). Emailed Oleg to ask for the code. —Theopolisme14:53, 21 December 2012 (UTC)
Normally that error indicates that the page is in categories for two different quality ratings or two different importance ratings. But, because of some corruption in the toolserver database, the double-ratings might be present on the toolserver even though they are not present in the live wiki. That can be verified by doing a query from the toolserver to see the categories that the talk page is in. If they do not match the categories visible on the wiki, the solution is to delete and undelete the talk page. Editing the page will not fix the corruption, in my experience. — Carl (CBM · talk) 16:03, 24 December 2012 (UTC)
The toolserver sysops have been working with the WMF sysops to try to get a fresh copy of the database, to eliminate all the corruption, but this has been delayed several times. In the longer term, if the WP 1.0 system ever moves to the up-and-coming Wikimedia Labs, that will probably solve many of the database issues. For the meantime, however, the toolserver database is the only way to get the assessment data quickly enough. Before we switched to that, updates were taking well over a week each, and that was years in the past, so they would take even longer now without using a database replica to read the assessment data. — Carl (CBM · talk) 16:06, 24 December 2012 (UTC)
Any idea when the fresh copy will be available? I've noticed similar issues with some of the other toolserver tasks, such as svick's cleanup list tracking and DashBot's unreferenced BLP lists. The-Pope (talk) 16:14, 24 December 2012 (UTC)
Was wondering if this or some other tool will generate a list of users who have edited pages identified to a project, sorted by most recent edit? Purpose would be so that the project could identify the currently active editors and/or project members. RiverStyx23{submarinetarget} 18:05, 6 January 2013 (UTC)
I have set up the category and run the bot for George Washington University Special Collections Research Center-related articles by quality statistics. The system says the contents have been generated, but the table does not appear. I think my project name is too long and is being truncated - is that the problem? If so, I can change it. - PKM (talk) 20:15, 1 February 2013 (UTC)
I gave the project a shorter name - George Washington University-related articles by quality statistics works now. - PKM (talk) 18:46, 2 February 2013 (UTC)
On the WikiProject assessment logs (like this one), would it be easy enough to add the name of the person doing the assessment? So instead of "Importance rating changed from Low-Class to High-Class", it would say "Importance rating changed from Low-Class to High-Class by User:Example". WhatamIdoing (talk) 17:42, 21 February 2013 (UTC)
If the change is because someone edited the page, then the edit in question can usually be found by looking at when the page was added to the category (in the catgorylinks table) with a join against the revisions table to pull out the editor's name. But there are two scenarios where this breaks:
If someone deletes the template, and someone else adds it back, the categorylinks table will show the date from when it was added back.
If the change was due to a template edit, there may be no edit on the talk page to match. For example, template Foo might categorize into "Category:Start-Class Foo Articles" and then someone may edit the template to use "Category:Start-Class Foo-Like Articles". Now the rating category will (eventually) update on all the talk pages even if they are not edited.
If you ignore those corner cases and just use the categorization date blindly, it should work most of the time. — Carl (CBM · talk) 12:08, 22 February 2013 (UTC)
Number of pages counted by the bot appears to be off by one
Hi, Illia! I've resolved the issue; the problem was that the bot hadn't run for that particular project in a few days, and, as such, the templates got 'out of sync.' In the future, you can rectify this issue by going to this page and entering the name of the project in question. Happy editing, —Theopolisme(talk)20:07, 2 March 2013 (UTC)
N/A importance book-class articles showing up as "Other-class" not "Book-class"
I just noticed on User:WP 1.0 bot/Tables/Project/Australia that there were 8 "Other-class" articles. These are all actually "Book-class" articles, but they each have importance=NA, so for some reason the Bot doesn't put them in the Book-class line, but thinks they are "NotA-class" (see the list). Can this be fixed? Thanks, The-Pope (talk) 03:39, 4 March 2013 (UTC)
In the case of Australia, I think this is a somewhat subtle error. They had two different "Book-Class" categories in Category:Australia articles by quality. The system will only take one. I think I have fixed it [8] but I need to wait for the enwiki replag on toolserver to propogate the category change. — Carl (CBM · talk) 19:05, 19 March 2013 (UTC)
Yes, that seemed to fix it. This problem comes up from time to time when someone tried to put task force categories into the 'by quality' category of the parent project. Task force categories need to be in their own 'by quality' category - the system only reads one "FA-Class", one "A-Class", etc. from each 'by quality' category. — Carl (CBM · talk) 14:24, 20 March 2013 (UTC)
WikiWork data
WikiWork data for tables for individual mathematics subject areas show the values for the entire mathematics project: Wikipedia:WikiProject Mathematics/Wikipedia 1.0/Table. This could be a bit confusing or misleading. Could you either calculate the statistics for each subject area separately, or suppress the data from the individual subject area tables? This may affect other projects as well. Illia Connell (talk) 19:19, 18 March 2013 (UTC)
How do you score B+ articles in the Mathematics project?
Did you consider incorporating the importance rating into the scoring system so that, for example, and high importance stub gets a higher score than a low importance stub. This may make comparisons across projects more meaningful; thus a project with a few low or mid importance start or stub class articles would get a lower total score than a project with many top or high importance B or C class articles.
Do you have a list of WikiWork Scores for all projects?
We currently use the standard formula that was defined back ~2007; this doesn't incorporate B+. I can easily add a handler for it, though. I'll just weight it slightly less than B (i.e., 2.5?). Would that be good?
No, I haven't considered that at the moment — currently, I'm just using the previous algorithm.
Just add an onwiki table generator to the code—the script should dump its output in table format at User:WP 1.0 bot/WikiWork/all by 12:00 UTC.
When a project add a new rating, it is set up in their ReleaseVersionParameters, e.g. Category:Mathematics articles by quality. There is an "extra1-replaces" field, which is already parsed and stored in the database, which gives one of the "standard" ratings that can be used used to replace the custom rating for encyclopedia-wide statistics like WikiWork. — Carl (CBM · talk) 12:48, 19 March 2013 (UTC)
I took a quick look at the WikiWork relative scores: See below for corrected figures
Regards, Illia Connell (talk) 00:58, 20 March 2013 (UTC)
Very interesting; thanks for generating that! I think it might make sense to at some point set up some sort of script to do this—I'll definitely add it to the todo list, if only for the chance to mess around with Python's imaging/graphing functions, which I've somehow avoided for so many years... ;) —Theopolisme(talk)01:26, 20 March 2013 (UTC)
Relationship between average WikiWork score and the number of articles in a WikiProject
See below for corrected figures
Not unexpectedly, there is a moderate positive relationship between the average WikiWork score and the number of assessed articles in the WikiProject. (I calculated the number of articles somewhat crudely as ω/Ω.) Regards, Illia Connell (talk) 22:43, 20 March 2013 (UTC)
These graphs are fabulous; however, there's currently a bug in the software (had to do with the API being rather cagey about its responses) that caused all previous WikiWork scores to be incorrect. Could you regenerate the graphs once the bug is sorted out? —Theopolisme(talk)22:49, 20 March 2013 (UTC)
Graphical displays of the relative WikiWork scores
Here are the updated figures based on the latest calculations. Click to embiggen.
If anyone is interested, the data and R code are here: User:Illia Connell/Graphical displays of the relative WikiWork scores/Data and User:Illia Connell/Graphical displays of the relative WikiWork scores/R code
Cumulative distribution of Relative WikiWork scores
Scatterplot and LOESS of Relative WikiWork Score and Number of Assessed Articles Note: Number of articles estimated as ω/Ω
WikiWork
Having project metrics is a very good idea. However, WikiWork is not a good metric at all.
The way WikiWork is constructed, lower ω means (or should mean) "better", higher ω means "worse", and the same ω means "no substantial change", but this is not always true:
If one creates an additional stub article, ω is increased, as if writing stubs is harmful.
If one creates an additional FA, ω stays the same, as if FAs are worthless.
Upgrading an article from Stub to Start is the same as upgrading an article from A to GA, while I would venture to say it's not nearly the same.
ω is inherently higher for big projects and lower for small projects, which does not make sense unless higher ω was "better".
One might abandon the idea that lower ω means "better", but what is the purpose of such metrics then, especially given the fact that it would be trivial to construct a metric that does not have any of the above flaws? GregorB (talk) 18:35, 19 March 2013 (UTC)
The idea is to document how much work is left to be done, not how much you have accomplished already. So if you create an additional stub, you have more work to do. If you create an additional FA, you do not. WhatamIdoing (talk) 18:37, 19 March 2013 (UTC)
Okay, WikiWork measures how much work is left to be done, but is higher ω better or worse? It appears that the answer is "it depends". My point is that if we can't tell whether more work (as measured by WikiWork metric) is good or not for the project, then the metric is not really actionable, as project participants cannot correlate their contributions with the WikiWork figures. GregorB (talk) 12:42, 20 March 2013 (UTC)
The idea with ω is to document how many assessment class changes it would take to achieve the nearly unobtainable goal of bringing all articles in the project up to FA-class. Thus, the ultimate goal would be ω=0. However, ω is a raw number that, as mentioned, is automatically larger for projects with far more articles than smaller projects. Therefore, you should not rely on ω alone to determine how the project is doing, as the raw ω won't really mean anything except that is how much work is left to be done.
This is where relative wikiwork, Ω, factors in. Ω = ω/N (where N is the total number of articles in the project) which gives the average rating of all articles in the project. Each time an article is reassessed to a higher class, the Ω will be reduced. Now, that reduction could be as small as going from say 4.056 to 4.054, but it is a measurable change that editors can see and correlate to a meaningful value--in this case, the improvement brought the average project article that much closer to C-class. Thus, the ultimate goal is also to get Ω=0.
Ω also helps address a couple concerns above. The addition of a stub does increase ω, but Ω factors in that a new article has been added and changes accordingly. The addition of a brand new FA has no effect on ω, but Ω would actually be lowered since an additional article is factoring in. -- LJ↗20:16, 20 March 2013 (UTC)
Ω might be called the "average quality" (lower Ω = higher average quality). In four cases I have enumerated above, Ω works fine in #1 (average quality is decreased), #2 (average quality is increased), and #4 (average quality is comparable), but still does not work quite right in #3 (same gain in average quality), albeit only on account of weights chosen.
where, say: a=100, b=40, c=40, d=40, e=15, f=8, g=3, h=1 (so 1 point is worth "1 Stub), i=3, j=0.5. This is meant to measure what has been done, rather than what is left to do, so higher is better. The weights are fairly arbitrary, but I feel that they reasonably correspond to actual work and/or benefit to the reader. As unassessed articles are Stubs at worst, they might as well be worth 1 point, but the lower value of 0.5 points provides an incentive to assess them. W behaves as it should in all four cases listed above. The average quality is W/N, and the absolute work that is left to do can be calculated indirectly as 100*N-W. GregorB (talk) 00:19, 25 March 2013 (UTC)
Looks like the bot skipped March 25th, felt guilty about it, and made two 26ths. ;) Just kidding..in all seriousness, I'll look into it. Probably just confused. —Theopolisme(talk)21:50, 28 March 2013 (UTC)
It probably means the replag was high enough on the toolserver that the bot did not know it had made that log entry already. The bot uses the enwiki_p.revisions table to tell when it edited the page, so if that table is sufficiently far out of sync the bot gets confused. The better way to avoid that would be for the bot to check the replag and not edit if the replag is too high; I never got around to coding that in, though. — Carl (CBM · talk) 22:06, 28 March 2013 (UTC)
Not updating
The bot is alleged to run every day. At this page there are said to be 15 unassessed articles. I have assessed every one of them (for example Talk:All Saints' Church, Stamfordat 12:07, 31 March 2013 )for this Stamford project. It still says 15 unassessed.
I tried running the bot by hand using http://toolserver.org/~enwp10/bin/update.fcgi, which said it completed and produced a green banner. That was a few days ago. I've just checked, and the 15 are still there.
Nothing, until the replag drops, I don't think the bot will run correctly. Keep an eye on the actual cats Category:Unknown-importance Stamford articles and Category:Unassessed Stamford articles rather than the toolserver's copy, and you'll keep your project fully assessed. The-Pope (talk) 16:16, 3 April 2013 (UTC)
There has been an enormous amount of discussion of that, but mostly on toolserver-related or WMF-related lists. The plan is for Wikimedia Labs to replace the toolserver. The WMF hired a person to work on this specifically, see [9]. — Carl (CBM · talk) 23:09, 3 April 2013 (UTC)
Help requested to diagnose problem
WP 1.0 Bot is awesome and I have used it before.
I am setting up a new WikiProject - WikiProject Dietary Supplements - and WP 1.0 Bot is not making a table for me. Could I have help diagnosing my problem?
Hi Blueraspberry--glad to see you're setting up the bot! Currently, the replag on the Toolserver is quite high, which in turns means that the database replicas that the bot uses to find and assess articles are at times out-of-date (see Template:Toolserver for the current details). You can see a slightly more detailed explanation in this recent thread on my talk page. In the meantime, hang tight! The bot should recognize the category in a few hours. —Theopolisme(talk)20:58, 9 April 2013 (UTC)
Reading through Blue Rasberry's comment above, probably the same problem, will try it again in a couple hours. Sadads (talk) 23:19, 11 April 2013 (UTC)
Piggybacking on this question, is there a timeframe for this issue to be fixed? Is the plan to hold off until Toolserver->WMFLabs migration is complete? There are discussions happening at WT:FOOD and Template talk:Project assessments, the outcomes of which are dependent on when/if the bot will resume its botlike duties. (I would very much appreciate if one of the bot-bosses could weigh in at the template discussion). The Potato Hose↘18:23, 20 May 2013 (UTC)
Yep saw your reply there (also this @Theopolisme: is a clever and elegant response to the notifications thing); my concern is the total inability to click through the template to see the category intersections (I'm doing it manually but grrr I am annoy). — The Potato Hose22:58, 20 May 2013 (UTC)
The toolserver has been extremely flaky lately. But it looks like the 'update' phase did run successfully on May 19. I am going to run an 'upload' job manually and see what happens. But if that works, it would also work from the crontab.
It does look like migrating to the WMF Labs is going to be the best long-term fix. I chatted with Coren yesterday on IRC and migrated my non-database-requiring bot jobs there. It was relatively straightforward. He says that database support should be close to completion. Once that is going, it will take a little work and experimentation to migrate the bot there. One of the problems will be migrating the enwp10 project database there - I think that will require some sysadmin help.
At the moment, the toolserver is working - the Tennis project has been updated just now. I don't have any way to tell if toolserver will be flaky again, and I don't know when labs will be ready to even look into migrating the WP 1.0 bot there. — Carl (CBM · talk) 23:22, 20 May 2013 (UTC)
Will the protect database be cleaned up at all before migration? Looking just at the projects starting with A I can see lots of projects with zero articles, renamed projects like AFL -> Australian rules football and Astronomical Objects -> Astronomical objects and likely duplication/incorrect names such as Algeria and Algeria-related. Then there are a bunch of projects listed under W for WikiProject, such as WikiProject Schools and WikiProject Cities, not under S or C. WikiProjects Canada, Japan & China are all listed as Foo-related, not just Foo. Does the existence of these non-projects (and I'm not talking about marked as inactive projects, but actually dead/incorrect names) or variations on the names affect performance or reliability at all? The-Pope (talk) 00:40, 21 May 2013 (UTC)
It doesn't affect the performance or reliability - a project with zero articles is very fast for the bot to update, and a project that doesn't exist in Category:WP 1.0 assessments takes no time at all. On the toolserver, there is no appreciable delay until a project has a few thousand articles, and I expect WMF Labs to be even faster.
I don't trust automatic deletion of things, so any cleanup does need to be done manually. But if we delete a renamed project, we would lose that copy of its old re-assessment logs, so I generally prefer not to delete them. It would be possible to tell the web system just to not show any project with zero articles, but I am afraid that might lead to other confusion, so I never did it. There are so many projects that reading the entire list is a bad idea anyway. — Carl (CBM · talk) 01:09, 21 May 2013 (UTC)
@CBM: Looks like the manual update.pl job worked, which is good. I'll keep an eye on the contribs--weird things have been happening now and then with my personal crontab on toolserver, so this might be related to that (??). Also, yes, I'll be happy to help with Labs migration (once I get my own tools over as well). Theopolisme(talk)10:57, 21 May 2013 (UTC)
I'm not sure what you mean by the service. The bot updates its data from the wiki and uploads tables back to the wiki - that part seems to be working. The web pages for the bot are entirely separate from the update/upload cycle. Even if the web pages are turned off, the bot can continue to update the on-wiki tables. — Carl (CBM · talk) 18:04, 21 May 2013 (UTC)
I mean, sorry, when you click through to category intersections on the template, you get a response saying service is disabled. — The Potato Hose18:14, 21 May 2013 (UTC)
I have re-enabled those now, since the toolserver seems to be slowly getting more stable. — Carl (CBM · talk) 19:55, 21 May 2013 (UTC)
Thought this really deserved it's own section. I understand that lots of people here get very concerned about things disappearing or not working, but I am struggling to fully understand why making the Project Index more accurate, complete and up to date is a bad thing. Surely we don't need to have "Electrical engineering", "WikiProject Electrical Engineering" and "WikiProject Electrical engineering" listed three times in the index? The comment about losing the log files is moot at the moment as no project's log is working on the tool server, and any actual former project should still has it's log history in EN:WP space, such as the AFL project (since renamed to Australian rules football). This list **could** be a really useful and valuable list for lots of other bots or tools, such as Svick's Cleanup list, DashBot's UBLP work or Dabsolver's project list, and maybe it is already used and cleaned up by them. But in it's current form it needs to be manually cleaned up a fair bit to be truly useful. And whilst I have no idea what goes on behind the scenes to make it all work even with the variations in displayed project name, I'm hoping that most of these changes are only cosmetic.
So can I suggest that:
all projects listed as "-related" such as "Canada-related" are stripped back to just the actual project name, ie just plain "Canada"
all duplicated projects due to capitalisation are removed to only leave the correct capitalisation.
Renamed, defunct, merged or zero article projects are removed or renamed only upon specific individual requests (with the appropriate checks that on-wiki logs/history exists if that is deemed worth keeping, or that it was only a proposed project that never actually got started, and not just a malformed project setup or spelling/capitalisation error). And being the nominator, I officially request that the AFL be removed from the list, as it is fully covered by WikiProject Australian rules football and by old on-wiki history files, and it's existence on this list is confusing, outdated and of no value. - The-Pope (talk) 15:54, 21 May 2013 (UTC)
I think that the images have simply been deleted. I might have missed the one you were talking about. Is there a page that exists (is not deleted) but the link to it is broken? — Carl (CBM · talk) 13:58, 4 June 2013 (UTC)
Yes, e.g. there is a link HNK Hajduk Split seasons ([[:HNK Hajduk Split seasons]]) which should have been Template:HNK Hajduk Split seasons ([[Template:HNK Hajduk Split seasons]])
It should be fixed now. Thanks for pointing this out, it was a bug that happened with the migration of the WP 1.0 bot from the toolserver to the WMF labs server. For other maintainters: to get the bot to re-upload logs, give it a date on the command line: copy_logs.pl PROJECT 20130601. — Carl (CBM · talk) 16:06, 4 June 2013 (UTC)
I learned this afternoon that the WMF Labs now has database replicas for enwiki, and they do support user databases, which means that it should be possible in principle to start the process of migrating the bot to labs. I will put an outline of the steps that I know of below this post. Please feel free to expand or annotate that outline. @Theopolisme: and @Wolfgang42:: I am willing to help work on the migration. I have moved my other bots to labs already, so I have a general sense of how to get things going there. Please let me know how much or how little you would like me to look into it. — Carl (CBM · talk) 20:18, 21 May 2013 (UTC)
By all means--your help would be very much appreciated. I'm a) busy and b) don't have much experience with Labs, so anything you're willing and able to tackle, go for it! Theopolisme(talk)21:02, 21 May 2013 (UTC)
Steps for migration:
Done Create a "service group" for enwp10 in the tool labs
Created the 'enwp10' service group at [10]. I can add anyone else to this group who has shell access to the tools project - see [11].
I have given you access to the enwp10 service group. You get to it by ssh'ing to the tools-login server and then typing 'become enwp10'. At the moment I am importing the database backup from toolserver into the database on labs. After that, I can work on migrating the code. — Carl (CBM · talk) 16:56, 23 May 2013 (UTC)
I have access--I think--with account Wolfgang42, shell name 'wolf'. I'm rather confused as to how the Labs permissions system is set up, though, so I'm not sure if I have the permissions I need. How would I check, and get them if I don't? — Preceding unsigned comment added by Wolfgang42 (talk • contribs) 20:20, 7 June 2013 (UTC)
I got everything sorted out, and now have access. I'll start getting selection_tools running over the next few days. — Wolfgang42 (talk) 14:52, 8 June 2013 (UTC)
Great. It does look like you have access to the enwp10 "service group". I copied over quite a bit from toolserver, but it is not arranged quite the same way. The selection tools code may already be copied. One other strange thing is that the database has a strange name based on the "mysql username" of the enwp10 user. You can find the name in .wp10.conf - it is too opaque for me to remember. — Carl (CBM · talk) 19:42, 8 June 2013 (UTC)
Done Copy the code from the current enwp10 directories on toolserver into the new directories on labs
Done Obtain a database dump of the current enwp10 user database on toolserver
Done Set up crontab on labs and disable it on willow
I have turned off the 'upload' task on willow, but left the 'update' task running for now. Both the upload and update tasks are running on labs. — Carl (CBM · talk) 14:22, 31 May 2013 (UTC)
Done This would be a good opportunity to remove the 'tmp' prefix from all the database table names
@Theopolisme: I think I will be ready to do an upload from the labs server in the next day or so. If that works, it should be possible to migrate everything by the end of the week, if you are ready for that. — Carl (CBM · talk) 23:17, 27 May 2013 (UTC)
At the moment, things are running on labs, and they seem to be working correctly. The main remaining thing to look at is web-based update. This should be better on labs because we can leverage the sge system to track an update request, rather than having to write a separate daemon for it. — Carl (CBM · talk) 14:22, 31 May 2013 (UTC)
Don't know if this is the right place to ask, but in the release version tools, would it be possible to add an option to sort by article size? That would be a great tool to find stub-class (and start-class) articles which have expanded and should now have a higher classification. --WS (talk) 07:54, 28 June 2013 (UTC)
While probably not of much use for your purpose, there is the long stubs report, but it gives no indication of their class. I will also note that size alone does not determine a stub. I believe that the largest valid stub has 37613 characters. Vegaswikian (talk) 19:15, 28 June 2013 (UTC)
Many of the longest "stubs" on the the long stubs have been changed by both the "class=stub" and the stub template. I proceeded to update some more. How often is this list updated? --DThomsen8 (talk) 19:57, 25 July 2013 (UTC)
To answer your second question. There is no tool, that I am aware of, that compares the project class determination on the talk page with the presence or absence of a stub tag. I will say that it can happen all too often. Also there are probably a large number of list articles that are not assessed as list class. Don't know how big of a problem this is. Both of these should not be fixed by a bot since the bot would have no way of knowing which one is incorrect. Vegaswikian (talk) 20:38, 25 July 2013 (UTC)
I have added thousands of talk page templates, and I would say there are many thousands of articles which should be class=list and not have stub templates in the articles, especially sports and entertainment (awards) articles. It is a big problem, but I am not sure it is an important problem. For me, the big articles on the long stubs list on science and history and politics are much more important and more interesting than the football ones. Those will be the ones I will occasionally fix. --DThomsen8 (talk) 13:41, 28 July 2013 (UTC)
An interesting observation about those large stubs. There are a number of science ones (genes) on there that are among the largest and are in fact stubs. While I have not counted, I suspect that we have some stubs where we have 10 or 20 characters of text in references for each character of article text. Vegaswikian (talk) 17:17, 28 July 2013 (UTC)
To answer my own question partially: One way of finding large stubs is looking for the stub template category, e.g. this way for medicine stubs. However this does not include stub-rated articles without the stub template. --WS (talk) 13:45, 1 August 2013 (UTC)
And a related question: is it possible to have a bot go through all articles within a project and set class to redirect for all redirects? I have encountered many stub-rated articles which have been turned into redirects; manually setting the class on all of them seems a big waste of time.--WS (talk) 13:49, 1 August 2013 (UTC)
WikiProject Antarctica
On 4 July 2013 the 1.0 Bot updated the assessment table for WikiProject Antarctica. Oddly, the bot removed all of the articles in the FA, GA, Start, Stub, Portal, and Template classes (diff [13]). Could somebody please take a look at this and reverse this bug? I suspect it might have something to do with the creation of a "NotA-Class" underneath the NA-Class category. Thanks. Altamel (talk) 19:12, 22 July 2013 (UTC)
Wikiwork list
Can any one direct me to any table or list containing the WikiProjects and their corressponding wiki work numbers? I remember seeing it but cannot find it now. —Indian:BIO · [ ChitChat ]07:02, 28 June 2013 (UTC)
Someone might want to take a look over that table. I sorted it by relative WikiWork, and things are a bit screwy. The entry for Michigan road transport is right, at 1.88. However, Software should be around 5.47, and Deaf should be around 5.04. Podcasting should be at 4.83. I think the bot isn't counting all of the appropriate articles in computing WikiWork values. Also, would it be possible to have the bot display values to 3 decimals instead of 2? Thanks. Imzadi 1979→11:43, 12 August 2013 (UTC)
I am glad to see the updated table that you cite, but the table I see in the menu for this discussion is an earlier one, which says at the bottom "This is the web interface of the WP 1.0 bot · Discussion page · Bug reports
Current version: revision 341, updated Thu, 6 Jun 2013 by cbm
New schema branch
Debug: PID 924 has handled -4 requests"--DThomsen8 (talk) 02:11, 27 July 2013 (UTC)
Hi - I can't solve the problem myself, but I'll try to reach the people who can. It may be that they're on vacation at the moment. Walkerma (talk) 13:24, 6 August 2013 (UTC)
Actually, I just checked, it does seem to be transcluding OK, it's reading the correct number of total articles for today (4,260,151), rather than 4,167,291 as seen in the 6th June version. If your copy of the page isn't showing that try refreshing the cache (CTRL-R on Firefox). Cheers, Walkerma (talk) 13:36, 6 August 2013 (UTC)
Today the total is 4,274,609. The confusing problem is the date is still shown as "Current version: revision 341, updated Thu, 6 Jun 2013 by cbm". When or how does this date ever change, given that the transclusion appears to be just for the table.--DThomsen8 (talk) 01:22, 18 August 2013 (UTC)
The report is listing 91 articles as NA class but in fact most have a start, stub or C classification. project is Bristol. Jezhotwells (talk) 12:42, 21 August 2013 (UTC)
Is there a way to force an update of the project summary table for a specific WikiProject under the new system? Inks.LWC (talk) 01:51, 7 August 2013 (UTC)
Hi, I'm continuing some of my research on article quality in Wikipedia (published a research paper on this at WikiSym this year, see link from my user page) and thought I'd improve my data gathering technique this time around. I've poked at some of the source code of WP 1.0 bot and it appears that both it and I basically take the same approach, but end up with different results. The current version of the overall table lists 1,271 A-class articles in total. If I search from Category:Wikipedia 1.0 assessments starting with all categories matching "by quality" and looking at any sub-category of those matching "A-Class" (branching down further if necessary), I only find 790 articles and 2 redirects. Is there some obvious mistake I'm making here? Cheers, Nettrom (talk) 19:31, 24 September 2013 (UTC)
Although it's not blocking my progress, I'm still wondering about this. If there is a better venue for this question, please do let me know! Cheers, Nettrom (talk) 21:57, 1 October 2013 (UTC)
Sorry for the slow response - I only check this page now and again these days. The totals are supposed to avoid double counting; some articles (particularly high quality ones like A-class) will often be tagged by multiple projects, but the bot knows this and is supposed to only count each article once. Therefore I suspect the most likely reason comes from the fact that A-class is not used by all WikiProjects, and that it overlaps with GA-class articles (a historical quirk - I'll give you the history if you're interested!). For example, one project may tag an article as A-class while another may tag it as GA-class. It's possible therefore that the bot is undercounting the A-class in such cases, at least in the subcategories. This is just an educated guess, but I would judge that therefore that the upper number is the more accurate one. Walkerma (talk) 05:25, 16 October 2013 (UTC)
Wikipedia:Version 1.0 Editorial Team/Song articles by quality log
My question is about score of GA articles of the Cycling wikiproject see here. And my question is: How it comes that a page like Ellen van Dijk has an importance_score of only 773 compared to 1225 of 2008 Giro d'Italia and 1262 of 2010 Giro d'Italia?
I see on WP:Version 1.0 Editorial Team/Article selection that the score of an article within a WikiProject without importance rating (like the cycling wikiproject) is only about how many times the page is visited (hitcount, internal links and interwiki links):
Overall article score = Importance_score + Quality score. (quality scores of these articles are the same (GA-class))
Hitcount:
Ellen van dijk had much more page views over the last year than 2008 Giro d'Italia and 2010 Giro d'Italia. (for a quick view compare the last 90 days of Ellen van Dijk views (6161) with 2008 Giro d'Italia (1752) and 2010 Giro d'Italia (2755) )!? Internal links and interwiki links:
I can't find the number of the incoming internal links and external links, but due to the fact that the article of Ellen van Dijk is translated in more languages (22 vs. 18 and 18) and has links on more wikipedia pages (405 vs.
42 and 51) I doubt that these incomming links can make such a large differences between the scores.
I'm interested in the status of enwp10's migration from Toolserver to Tools as it's one of the bigger projects. I've found this archived discussion, but some questions remain:
On Toolserver, there is an SGE job running (probably launched from a crontab?):
timl@yarrow:~$ qstat -u enwp10
job-ID prior name user state submit/start at queue slots ja-task-ID
-----------------------------------------------------------------------------------------------------------------
3379217 1.50028 enwp10-upd enwp10 r 12/22/2013 13:27:08 medium-lx@yarrow.toolserver.or 1
timl@yarrow:~$
Am I right to assume that both are obsolete and can be closed?
I've searched for external links to toolserver.org/enwp10 on this wiki, and if there were ever any, they seem to have been converted to tools.wmflabs.org :-). Is a redirect rule toolserver.org/enwp10/$OPTIONAL → tools.wmflabs.org/enwp10/$OPTIONAL sufficient, or do we need to cater for any special cases?
You have a project "ENWPONE" in JIRA with several unresolved issues, 29 issues in total. Do you have plans on where to host/migrate your bugtracker in future?
I wonder if someone who knows more about the bot and its' page updates could have a look and let me know if they can tell why it quit updating. Thanks. N2e (talk) 18:35, 16 December 2013 (UTC)
Seems the WikiProject Tennis assessment table hasn't updated in a while and it doen't reflect the current status.--Wolbo (talk) 23:55, 28 December 2013 (UTC)
Not sure if something needs to be updated on this end, but it seems to report pages assessed as Draft-class as "Other" and "NotA-Class". ☺ · Salvidrim! · ✉03:16, 4 January 2014 (UTC)
The bot malfunctioned(?) at the WP:BEYONCE wikiproject assessment here. Basically it removed all the assessments of FA, GA etc and kept only the summaries. It might have to do with the recent "Move" of Beyonce Knowles and its related articles to just Beyonce. I have restored to the last good version. Can anyone check this please? —Indian:BIO · [ ChitChat ]06:38, 2 February 2014 (UTC)
@IndianBio: it was not a malfunction of the bot. Rather, it was a malfunction of the talk page banner for the wikiproject. Someone moved ((WikiProject Beyoncé Knowles)) to ((WikiProject Beyoncé)), but did not move the associated ((WikiProject Beyoncé Knowles/class)) subpage. Without that, the banner was no longer assessing for class, just importance. Now that I've moved the subpage, the class assessments should be restored as the server updates the pages again. Once things clear up, the bot will restore the assessments. Also, if the assessment categories are renamed, the table and log pages will also have to change names to match. Imzadi 1979→07:08, 2 February 2014 (UTC)
Well, yeah, it's possible. But it will require creating new assessment categories following the general format of Category:FL-Class Speed skating articles as well as updating documentation/tons of pages that will need to be recategorized and such. Personally, I'm not sure if it's worth it, but does the WikiProject see a need for this? Theopolisme(talk)23:06, 26 February 2014 (UTC)
Are the historical values of the Wiki work factors stored anywhere. Also can you make it show only the work for Top and High or Top High and Mid importance? I'm looking at the Economics projectJonpatterns (talk) 11:50, 12 March 2014 (UTC)
Hello! Recently, the parameter |Animated= was added to the ((WikiProject Film)) template, in order to include articles in the Animated films work group. I have started to add the parameter to talk pages, however the articles are not being added to the project with the proper classification, since the Film project does not use the |importance= parameter. Therefore, they are showing up on the assessment table for animated films as "Other". I understand that this one is unique, because it's technically a work group of the Animation project, but none of the other Film task forces use importance. Is there any way to have these articles default to the importance for the ((WikiProject Animation)) template? Fortdj33 (talk) 17:33, 8 April 2014 (UTC)
It's been a few days, but I thought that I would follow up on this. The concern is really just how the data is displayed on the assessment table, which still has an "Other" column that is unnecessary. Therefore, if a script can simply be added to modify the display of the table, no additional parameter on the project banner would be necessary. For example, something that would make film articles default to the value of |film-importance=, if present on the ((WikiProject Animation)) template, and default to "NA" if the parameter is not present. Can it be done? Fortdj33 (talk) 12:44, 11 April 2014 (UTC)
That was my initial suggestion, but as I said in my follow-up, what really matters is how the articles are displayed on the assessment table. So if an article has both the ((WikiProject Animation)) and ((WikiProject Film)) banners, the table should reflect the value of the |film-importance= parameter on the Animation banner. Otherwise, a value of "NA" would be preferable to "Other". Fortdj33 (talk) 16:23, 11 April 2014 (UTC)
Also, if this is not the right place to inquire, could someone point me to the correct location? Any appropriate direction would be appreciated. Thanks, -- Black Falcon(talk)17:01, 12 April 2014 (UTC)
Exporting article lists as a csv file
Hi, I'm trying to create a spreadsheet with the article titles of the Top-Importance articles on all of English Wikipedia sorted by Score. I have this query
Great question @Ocaasi:. Seems worthwhile to add a request on WP:Bugzilla to have a "download" link added to that screen -- I have wanted this in the past too. -Pete (talk) 18:06, 2 May 2014 (UTC)
Actually, Pete, this is a volunteer-driven project without a Bugzilla presence. So, Ocaasi, your request has been noted; if I have some time, I'll see what I can do. Theopolisme(talk)03:29, 3 May 2014 (UTC)
Looks like everything was migrated to a new server and broke in the process. I've emailed the mailing list; hopefully someone there can provide guidance. I tried following their "migration instructions" but they didn't seem to work... Theopolisme(talk)11:06, 20 March 2014 (UTC)
OK, thanks for looking into this and letting us know. Unfortunately I had to use the old URL in a publication I submitted recently. Any word yet? Walkerma (talk) 13:01, 22 March 2014 (UTC)
There were never any helpful responses. I ended up spending around 3 hours figuring out how to make everything work again; it involved rewriting configuration files, numerous restarts, etc. This is due mainly to the poor infrastructure and huge lack of documentation on Wikimedia Labs. tl;dr migrated datacenters, disabled all crontabs, changed how webservices worked... *sigh*. Everything should be working now, though! Please give it a try, Walkerma. :) Theopolisme(talk)15:03, 22 March 2014 (UTC)
It is still down, and has been for at least a week. For example, the link at the top of this section gets a 404 message. RockMagnetist (talk) 14:36, 10 May 2014 (UTC)
Jade Etherington was missing, and I highly doubt that there was only one GA yesterday. I have suspected for a few weeks that the bot is playing up a little, but I'm not sure. Thanks, Matty.00708:29, 17 April 2014 (UTC)
I see. I think that's a malfunction of the bot. Here's some more details I found: the bot used to post each day's log on the following day, but since March 23, it is posting the daily logs on the same day, that is, 6-7 hours after the day started. I can imagine that it now misses all activity that happens on any day after the bot ran. For example it missed Jade Etherington, who was promoted to GA at 10:42 UTC.
Yes, that fits: the article was deleted at 9:27 and the log was written at 6:24. We need to wait for the operators to set the bot back to compiling each day's log on the following day rather than the same day. — HHHIPPO13:44, 18 April 2014 (UTC)
@Theopolisme: No problem, and thanks for the fix! I actually think my error description wasn't complete, but I'll wait for a few bot runs before investigating further. — HHHIPPO21:47, 27 April 2014 (UTC)
Here's two problems I can see at the moment: (1) as you're probably aware of, the online tool is currently unaccessible. (2) While the bot is now posting each day a list of activity that happened the previous day (as it should), it is labelling these lists (both in the section header and in the edit summary) with the dates the lists are posted, not with the date on which the listed edits happened. (What's described as "Log for April 30" is actually the log for April 29, posted on April 30.)
I can't tell yet if the lists are complete. Having the online tool back would be great for comparing, but I'll also try to test some samples against other tools. — HHHIPPO20:42, 30 April 2014 (UTC)
@RockMagnetist, PrimeHunter, and Czar: Hi everyone! Sorry for the delay -- I've been quite busy with real life and didn't see this until now. The server the bot and webservice run on is WMF-operated and out of my immediate control, so I was a bit mystified by this problem as I haven't done anything with the bot for a while. Thankfully, after an hour or two of investigation, I believe that I have fixed the web tool (yay!)...still investigating reports about problems with the on-wiki updates, though. Theopolisme(talk)21:18, 13 May 2014 (UTC)
I'm hoping to clarify a few things in regards to bot and web service outages over the past few weeks.
WP 1.0 bot and the associated web service run on Wikimedia Foundation-owned and maintained servers (Wikimedia Labs).
I (as the tool) have an account on Labs; I am able to run programs there, so that they execute on their servers 24/7 as opposed to only when my machine is on and connected to the internet.
Not only were scheduled tasks deleted, some changes were made to the servers that meant reconfiguration and restarting were necessary.
For a period of about a week or so, WP 1.0 bot did not run simply because its scheduled tasks were deleted. After I was alerted about this problem, I began investigating the issues, but with limited time and resources (WP:REALLIFE).
I was able to restart the web service a day ago and tools.wmflabs.org/enwp10 is once again functional.
I have just reconfigured and restarted all scheduled wiki-editing tasks: table updating, log updates, and the like. These will probably take a bit of time to catch back up -- if something is still not being updated in a few more days, please ping me to ensure I receive the message.
Hi Theopolisme, I'm currently looking for an API acccess to integrate individual page assessment data into . Possible?
In addition I can offer you a more resilient webservice setup on toollabs, including a webwatcher that restarts webservice on failure. Tested & approved with Xtools and other tools, serving >100,000 request without stalling. see [16]
USRD and the other highway projects are using AL-Class, but when our first list was promoted at ACR, the bot logged that the page has been removed. Can someone apply whatever fix is necessary to recognize this class? Thanks, Imzadi 1979→09:12, 5 June 2014 (UTC)
Same for the Ireland WikiProject. Nothing since 12th June. Can we get some sort of communicationas to what is going on? ww2censor (talk) 18:25, 26 June 2014 (UTC)
Echo the above for WP:Rivers, taking a leaf out of a previous thread - I will ping the operators @Theopolisme: and @Wolfgang42:, hopefully they can restore normality to these useful tables, and their not dead - just resting...Jokulhlaup (talk) 17:16, 27 June 2014 (UTC)
That page was returning a 404 error when I posted the original post, along with all the links to Labs from the table. Looks like that's now sorted so hopefully the bot will be back up and running soon. WaggersTALK07:33, 3 July 2014 (UTC)
It looks as if Theo has been on holiday for the last couple of weeks - he has no contributions since 20th June. I'll see if I can raise someone else to work on this! Walkerma (talk) 02:09, 9 July 2014 (UTC)
Apparently it's a fairly general issue (server?), and it's affecting several of the tools on the WMF server. That means it's beyond our control - unless someone has a backup server we can use! I'll keep pestering people and I'll post here when I hear anything more. Walkerma (talk) 13:32, 11 July 2014 (UTC) It does seem to be updating things today, at least. Walkerma (talk) 13:39, 11 July 2014 (UTC)
Tool not working for the past two months
Can we get an update; has it been abandoned? None of the links on the tool server are working and I cannot submit an update request. It would be a shame! - Sweet Nightmares14:10, 14 July 2014 (UTC)
Indeed. Oddly, the articles by quality logs are still regularly updated, so I guess the bot is not completely down. GregorB (talk) 18:18, 14 July 2014 (UTC)
There is something going on in the web server query part of the labs port; I do not have commit access to the tool anymore, so I can't really go around to try to poke and fix it. We need to wait for either Theopolisme: and Wolfgang42 to show up, or ask someone else familiar with the perl side of things to help. Titoxd(?!? - cool stuff)03:44, 15 July 2014 (UTC)
Hi guys. I've been really busy with work outside of Wikipedia, but I set up a daemon to automatically restart the webservice whenever it crashes (the crashes are not caused by us, but rather by issues with wikimedia labs where the site is hosted). The site should be up now. My apologies for all the trouble it's been having lately! Theopolisme(talk)22:26, 16 July 2014 (UTC)
@Theopolisme: No worries, I totally understand, I was just checking for a pulse is all. :) Glad to hear it's up and running again; looks like everything is working swell. Thanks so much for all the work you've put into this! - Sweet Nightmares03:08, 17 July 2014 (UTC)
Completely missed the other two discussions. Big "DUH!" on my part. Thanks for running the bot manually and for the link, much appreciated. :) - Neutralhomer • Talk • 11:53, 21 July 2014 (UTC)
@TeriEmbrey: I ran it manually and it updated. Javascript has to be enabled in the browser when selecting the project on the manual update page so that it can copy the project name to the text box. --Bamyers99 (talk) 16:25, 20 August 2014 (UTC)
At WP physics, WP 1.0 bot has added the logs for October 22, 23, 24, 25 and 27 twice, but never the one for October 26. I'm not sure if that's a problem, and I didn't check if there was anything to log on the 26th or if it's maybe included with another day. Just thought I'd let you know in case you want to check if there's something serious behind this. — HHHIPPO22:19, 29 October 2014 (UTC)
WikiWork factor cross-project comparison
Are there any tools for large-scale comparison of WikiWork factors within the bot? I'm currently looking at reviving a WikiProject, and am intrigued to see what comparison of a project's Ω and total articles yields. I have a hypothesis that 'model' WikiProjects are more likely to have a lower Ω when taking article numbers into account, which would help in identifying WikiProjects more successful in promoting articles, but am currently unsure how to confirm this using data from the bot. — Sasuke Sarutobi (talk) 23:31, 4 November 2014 (UTC)
@Sasuke Sarutobi: Not sure if I'm misunderstanding your question, but Ω already takes article count into consideration. ω is the raw aggregate number of classes, and is divided by the number of articles assessed by the project to obtain Ω. If you're trying to see if Ω is correlated with article count (i.e. if a high article count tends to be associated with lower Ω), then I don't think the bot can help you; you'll have to obtain the raw Ω and article count scores for a large sample of projects and use something like Excel to compare them. —Scott5114↗[EXACT CHANGE ONLY], creator of Ω, 09:48, 14 November 2014 (UTC)
Assessment logs
The assessment logs are inaccessible due to what I presume is a bug. I would have reported it as such but the bug reports page returns a 404 error.
Select any letter and click the "log" link in the data column.
Expected behavior
The project's log should be displayed.
Actual behavior
The error message "Project '[name]' is not in the database," even when the project name works just fine for things like the article list or assessment tables, proving it is, in fact, in the database.
I do hope this will be fixed soon; I need access to the assessment log in order to create some assessment graphs (manually poring through the on-wiki log will take too much time since I need to go back through February). —Scott5114↗[EXACT CHANGE ONLY]06:19, 30 October 2014 (UTC)
WP India - Template category wrong pointers, counts
As we embarked on a Tag and Assess mini-drive, I found that the statistics shown at WP:IND had a problem with "Template" class row. Here is the bulleted list of observations:
(b) Template counts showed 28 under NA importance and 2 under Unassessed importance. The actual links lead to the Wmf Labs links NA Class count and Other count.
(d) The Wmf labs pointers and/or Category counts of the project are messed up, only for Template class India articles.
(e) Please see citation : WMF Labs - Template class list from English Wikipedia Category. 2014-11-15. Accessed: 2014-11-15 (Archived by WebCite® at http://www.webcitation.org/6U7R8w2My) The list of articles are not from Template-Class India articles, but from Template-Class football in India articles. The entries in the form at the top of the page were working fine. Now the "link" / "foreign-key" or some such relationship seems to have been messed up. It is probably that a small mistake during the "Football in India" project creation has caused this.
My manual update of the India project statistics has been overwritten by the bot run on 16th November (which is first time it has run in about 5 months - after mid-June 2014).
Hence Template link again points to the "football in India" sub-project, instead of India templates category!
Has this discrepancy been investigated before? Back in April I scribbled down some related observations and theories; although it was outside the 99% confidence interval, it wasn't obviously absurt and it was plausible that there wasn't a strict error anywhere—i.e., the bot was just counting differently to me. Now, however, it's definitely either counting some non-articles as articles or some articles twice. (My hunch is that it's redirects with assessment being counted as articles, as might be created by page moves, but that really is just a hunch.) Facing the Sky (talk) 14:35, 26 November 2014 (UTC)
Help creating new table for WP1.0 bot
I'm working on a new page for my WP:GLAM/Johns Hopkins University project, and I would really like to include an article assessment table. I think I've done most of the steps, but I can't quite figure out how to generate the table itself. Could someone help me with this? Thanks! archivist 20:55, 1 December 2014 (UTC)
Feature Request can I have an API?
I would like to treat the output of this bot as an API. I would like to build a bot that does specific quality analysis on Medical project pages, and I would prefer not to replicate the functionality already available in the Release Versio Tools Bot. In order to do this I would need to modify list2.fcgi to simply export json and/or csv instead of an html table in response to queries. I would configure a form checkbox to enable this mode from the gui permitting it as option...
There could be another way to get this done that I do not know of and if that is the case I would be more than happy to be answered with a link.
Would a patch to do this be accepted? Any chance of moving this to git for better distributed development?
@Ftrotter: This may not actually be helpful depending on what you're planning to do, but I saw the question in passing and I couldn't resist: scraping the bot's wikicode tables is a 100% feasible way to do historical analysis of article quality. File:Wikipedia article assessment graph.svg has BSD-licensed Python to do precisely that for the overall table, and it's not even all that horrible! Facing the Sky (talk) 14:56, 26 November 2014 (UTC)
@Facing the Sky: That looks very similar to what I want to be able to do. Essentially, I don't need an API b/c this tool writes back to a wikipedia page, and wikipedia itself has an API. Will move forward with this approach. Thanks for the suggestion! --Ftrotter (talk) 07:36, 5 December 2014 (UTC)
Table updates
As the bot is still not updating the project tables I have been using the form to create the table. But the form appears to be not been shown. Going here now just give a blank page. Is there a problem with the form and when will the bot be updating the table as it generates the logs again? Keith D (talk) 00:11, 28 November 2014 (UTC)
Actually the bot has been misbehaving for almost 6 monthssince at least 12 June last when it stopped updating the assessment tables but at least we were able to run it manually. Now we have nothing, no assessments at all and no one saying they are working on it. This problem really needs to be addressed. ww2censor (talk) 11:08, 5 December 2014 (UTC)
Other than Wolfgang42, the other two editors have not being active here for around a year, so may not have seen the pings. I have emailed them directly and hope at least one of them responds. Theopolisme is also on Freenode and IRC so maybe someone can make contact there. ww2censor (talk) 11:54, 6 December 2014 (UTC)
I am around sometimes, but I am no longer a maintainer of the bot. Please don't ping me for it. Thanks, — Carl (CBM · talk) 18:39, 6 December 2014 (UTC)
I just got an email from ww2censor. I haven't looked at the 'bot since it moved to WMFlabs (in fact, that bit of the tool I've not looked at at all), so it may take me a while to get up to speed, but I'll see what I can do. — Wolfgang42 (talk) 19:14, 6 December 2014 (UTC)
@Ww2censor, Adamdaley, Wolfgang42, Keith D, and Brirush: & others... Hello everyone! My humblest apologies for my lack of response (and inaction) for the past few months. For the past couple of hours I have been investigating and getting the bot back up and running. The webservice had crashed (as was clearly apparent to end users!) a few weeks ago, and so my first project was to get the site online. We are now using the webgrid lighttpd daemon combined with trusty and bigbrother, which should keep the site from going down in the future. Next issue was the fact that for some reason the bot was no longer updating tables on-wiki, *but* (as I discovered in my investigation) data was still being updated on the server. It turned out that a change in the API had tripped up the bot (since for some reason I missed a notification about said changes...). I updated the relevant API calls, and the bot should now be uploading tables/logs as scheduled. Please let me know if any other issues are encountered; it may very well be that there were other changes that I missed... Again, my apologies for the inconvenience! Theopolisme(talk)21:25, 6 December 2014 (UTC)
Thanks Theopolisme. Manual updating is working fine for the projects I tested but don't know about the auto updates. I presume they will be ok now too. ww2censor (talk) 23:28, 6 December 2014 (UTC)
@AshLin: I happened to notice that this index page itself gives the link to instructions for setting up statistics for any project, towards the end of the page. Quote - If you would like to add a new WikiProject to the bot's list, please read the instructions - unquote. Hope this helps. VasuVR (talk, contribs) 02:13, 18 December 2014 (UTC)
New Project GLAM/Newcomb Archives and Vorhoff Library
I've created the categories for my new project (see Category:Newcomb_Archives_and_Vorhoff_Library-related_articles) and tagged a test article, but I can't seem to get the assessment table to generate. This is the first one of these I have done since before we changed the Toolserver, so maybe I've done something wrong. Can you advise? Thanks! - PKM (talk) 00:26, 16 December 2014 (UTC)
Yep. This is the error message at the bottom of the screen. "Project 'Military_history' is not in the database." The underscore syntax works for the Article lists tool, but not for the Assessment logs one. Peacemaker67 (crack... thump) 05:13, 3 January 2015 (UTC)
Well in the drop down menu "Military_history" is listed, so I manually ran it (http://tools.wmflabs.org/enwp10/cgi-bin/update.fcgi) as well as also manually typing "Military history" (without the underscore) and on this page history both assessments I ran clearly show up as 12:10 and 12:15 today plus the assessment log is being update regularly as you can see. In that regard everything looks fine to me. However the result table itself appears to be missing many entries that I see in other assessmeent logs I watch as it just gives total pages without any importance rate listings. Are you sure you are watching the correct page? ww2censor (talk) 11:19, 3 January 2015 (UTC)
Indeed, that is a diffenet thing. I never saw or used that tool but looking at the project list at the top left, I see "Military history" as well as other projects I am interested in are all listed in the database but putting their name into the form to generate the assessment says none of them are in the database. Sorry can't assist on that one as it seems like a technical issue. Maybe @Theopolisme: who fixed the non-working assessment log tool recently can assist though it may be best to email. ww2censor (talk) 12:07, 3 January 2015 (UTC)
This actually is a significant issue. For example, MILHIST cannot complete its quarterly reviewing awards process without this information. Can we get a fix on it, please? Thanks, Peacemaker67 (crack... thump) 01:11, 11 January 2015 (UTC)
Did you make contact with Theopolisme as suggested? I suspect he is the only one who can help but you need to email him. ww2censor (talk) 10:47, 11 January 2015 (UTC)
You are correct that http://tools.wmflabs.org/enwp10/ is blank again even though the individual project assessment tables are still being created. However, links from the tables also go nowhere either. Previously it was completely broken, except for manual runs, from about June to Novemebr last year but during that time it did not even automatically produce project tables. I think we need to email User:Theopolisme again. ww2censor (talk) 22:33, 26 January 2015 (UTC)
Restarted & should be live again... thank you for emailing me, Ww2censor. I don't know why the service was not automatically restarted, as generally when it crashes like this, the auto-restart mechanism I configured kicks in and "wakes it back up" automatically... in any case, let me know when/if other issues arise. Theopolisme(talk)03:44, 27 January 2015 (UTC)
Another maintainer as backup?
It would be good to look for another maintainer as backup (if anyone is willing or has connections to willing programmers). Please note, that I am not criticizing Theopolisme (or anyone else) in any way - after all this is a voluntary project and his help is greatly appreciated. But it would be better to have at least 2 (or even 3) maintainers for such a huge and widely used project, especially when the project or its environment aren't completely stable. Just some food for thought, while we wait for the fix :). Thanks again to Theopolisme and the other maintainers for their efforts. GermanJoe (talk) 22:49, 26 January 2015 (UTC)
@Doc James: The Toollabs database is currently about 6 hours behind in receiving updates from the production database. Any changes made to the production database will not be seen by the stats bot until the Toollabs database catches up. Current Toollabs database delay. --Bamyers99 (talk) 02:11, 31 January 2015 (UTC)
Updated after another manual bot check - seems to be OK now :). The bot probably got confused with the initial hyphen renaming and category title movements - but that's just a theory. GermanJoe (talk) 05:04, 31 January 2015 (UTC)
I sent a mail to Theopolisme (planned to send one anyway) to point to this problem. 2 categories are missing (see warnings on Template:WikiProject Insects) and the category naming is a bit non-standard (most task forces don't include "... task force ..." in their "project" designation for statistical purposes), but I am not sure that causes the problem. GermanJoe (talk) 17:49, 14 February 2015 (UTC)
Hello @Jonkerz:, to get the bot started, the 2 main categories (... by quality and ... by importance) had to be added to "Category:Wikipedia 1.0 assessments". I added them and ran the bot manually - it collected Ant task force assessments (please double-check, if all classes are filled correctly). You should still add the 2 missing class categories (see above), even if you don't plan to fill them, and get "Wikipedia:Version 1.0 Editorial Team/ant task force articles by quality statistics" deleted (the bot uses the uppercase "...Ant..." page). GermanJoe (talk) 10:35, 15 February 2015 (UTC)
User:EngineeringGuy has reverted the freeze claiming that 'The table has to be updated daily. "Freezing" it is not going to fix any problem.' Of course, the table does not have to be updated, let alone daily. As for fixing problems, freezing obviously does not fix problems per se. It simply works around User talk:WP 1.0 bot/Tables/OverallArticles#Modifications_will_be_lost. Anyone willing to actually fix the bot is most welcome. I have restored the workaround meanwhile. --Chealer (talk) 00:57, 8 April 2015 (UTC)
WikiProject Video games and eSports Task Force
Two Things:
One
The assessment table for Wikipedia:WikiProject Video games doesn't seem to be updating, and hasn't seemed to be updated for weeks. Take a look here [Category:Unassessed video game articles] the numbers don't match up
It's surprising nobody has used it as the basis for a daily updated wiki app. Sort of like how there's already a weekly featured article report — Preceding unsigned comment added by 82.6.237.36 (talk) 22:16, 20 May 2015 (UTC)
Project counter ?
Probably not the most vital statistic, but could someone have a look at User:WP 1.0 bot/Data/Count please? Looks like the bot stopped counting in mid-2013 for some reason. The count is used on this talk page's project page and is currently empty. Thanks for your help. GermanJoe (talk) 06:48, 18 December 2014 (UTC)
@Theopolisme:, when you have time, could you look into this minor problem please? The project count is still maintained as "Project index" on the tool's page, it just doesn't get uploaded anymore. GermanJoe (talk) 10:45, 15 February 2015 (UTC)
WikiWork factors
New Zealand politics articles by quality and importance
We have a couple of WikiWork factors and they are nicely explained. All good. I wonder whether we could or should have a third metric that deals more explicitly with the proportions of articles that are rated stub class. The Stub Contest gives some ideas why it might be good to expand stubs to something more useful for users of Wikipedia. To that end, I suggest we should consider a metric that directly shows the proportion of stubs. I'm thinking of summing up all articles from FA down to Stub (i.e. not including list or book classes or the likes). The number of stub articles divided by that sum would give a percentage stub articles of the total. This would be between 0% (no stubs at all) and 100% (i.e. every article within a project's scope is a stub). Thoughts? Schwede6620:38, 19 June 2015 (UTC)
Ok, I'll explain what I mean based on the assessment table shown. I'll make some calculations, and these refer to the numbers in this version of the table; the table on the right is dynamic, i.e. it will update itself over time and you could no longer understand the numbers that I'm talking about, hence the link to a particular version of the assessment table.
There are currently two WikiWork factors shown: ω and Ω. The first one counts the number of articles that have a quality rating of relevance, i.e. it excludes things from List class down. The second counts the number of quality improvement steps that those articles would have to go through to all become FA class. The number of steps divided by the number of articles gives omega. That is, it looks at it from the FA class perspective. The aim is to all articles to FA class. If a Wikiproject had no stubs whatsoever, you could not determine that by looking at the omega value. What I am proposing is to look from the other end. How many stubs are there as a proportion of all articles with a relevant quality rating? This would be expressed as a percentage, from 100% (all articles within a Wikiproject are stub class) to 0% (none of the rated articles within a Wikiproject are stub class).
I've worked through an example. When I use the figures in the table that I have referenced, I get slightly different figures (ω = 17,933; Ω = 5.279), but that may be because those values get calculated at a different time that the assessment data are compiled. Based on those values, the "stubbiness" would be 42.9% (1,456 stubs divided by 3,397 rated articles). Let's assume that we had a blitz on stubs, and 100 stubs were improved to start class. Resulting values are: Ω = 5.250 (i.e. it hasn't moved much) and "stubbiness" = 39.9%. The closer you get to getting rid of the stub articles, the less meaningful the omega value becomes, as it measures the number of steps to FA class. But with the "stubbiness" percentage, you have a good indicator how a particular project is going. Makes sense? Schwede6617:41, 1 July 2015 (UTC)
@TeriEmbrey: Are you referring to the index of participating Wiki projects [23]? Pritzker is listed as "Pritzker Military Library-related" - or did you mean a different list? You can also search for "Pritzker Military Library-related" in "Project summary tables" and "Article lists" - both seem to work OK. GermanJoe (talk) 22:10, 30 June 2015 (UTC)
@Ww2censor: There's a Draft:Volunteer Ireland with a talkpage banner causing this entry. As far as I understand the problem, draft namespace is not fully supported by the bot. The bot can manage drafts as extra class, if that setting is configured in the assessment categorization (see drafts in WP:GER assessment table). But the bot's statistic views and lists don't know about drafts as separate namespace and will treat such pages as "regular" article links. GermanJoe (talk) 17:25, 17 July 2015 (UTC)
Can we get the code that generates the Version 1.0 Editorial Team tables changed to fully support "Draft" class (as it does the other extended classes - Category, Disambig, File, Portal, Project, and Template, and as template AbQ does above) instead of lumping drafts into other? That's what you're seeing, Ww2censor. Stick "Draft:" in front of the name of the errant page and you'll find it, e.g. Draft:Volunteer Ireland. The result that disappeared was most likely approved at AfC and moved into article space, or it may have been deleted or reevaluated as not in the scope of WikiProject Ireland. Worldbruce (talk) 17:34, 17 July 2015 (UTC)
That answers the questions. Thanks. Perhaps the code will be fixed so such draft pages get classified correctly. Either way I'll know if I see some again. ww2censor (talk) 21:24, 17 July 2015 (UTC)
BTW, the problem was due to a recent file system failure at the Tool Labs. Everything seems to be in working order now, manual update works, so I'd like to see daily updates continue too, I suppose nothing is preventing them now. GregorB (talk) 18:23, 26 June 2015 (UTC)
That's odd - true, you typed a lower case "c", and the page got created by error, but still these things should not happen... GregorB (talk) 18:37, 28 June 2015 (UTC)
I have found two articles that are non-existent, but are actually being counted as being articles. The first one is. The Windows Club being in the WikiProject Microsoft Windows section. The other article is in the WikiProject Military History Biography called Robert LaRue Miller. Unfortunately, in either one there no article exists. They are however countered towards the table statistics. Adamdaley (talk) 06:58, 23 July 2015 (UTC)
@Adamdaley: Draft:The Windows Club and Draft: Robert LaRue Miller both exist. The bot has not been updated to support Draft-Class and the Draft: namespace correctly, and when it updates the assessment logs, it omits the namespace portion of the draft titles. Imzadi 1979→07:30, 23 July 2015 (UTC)
Here's the Robert LaRue Miller article: [25]. Basically the same as the other one in the statistic table. Except in another table and different Project and article name. Adamdaley (talk) 07:42, 23 July 2015 (UTC)
@Adamdaley: let me try again to explain the situation. The Draft namespace was created in December 2013. At that time, the bot was not updated to account for it. When I tagged File talk:Interstate 696 pedestrian plazas Oak Park.jpg for the U.S. Roads WikiProject, the bot saw that new assessment. It knows about the "File" and "File talk:" namespaces, so when it added that new assessment to the USRD log, it linked to File:Interstate 696 pedestrian plazas Oak Park.jpg. The bot also knows about "File-Class", so it could log it as the correct assessment.
Now, when someone added the six wikiproject banners and assessments to Draft talk:Robert LaRue Miller, the talk page that corresponds to Draft:Robert LaRue Miller, the bot tried to process that assessment like it would for any other talk page. It doesn't know about "Draft:","Draft talk:" or "Draft-Class", so it gets confused. The bot drops the "Draft talk:" prefix from the talk page and doesn't replace it with "Draft". So instead of linking to Draft:Robert LaRue Miller, it links to Robert LaRue Miller, a redlink on the assessment log. At the same time, the bot can't handle Draft-Class as an assessment, so it logs it as "NA-Class"
In short, you don't have non-existent articles for those projects, you have mis-linked and mis-assessed Draft pages showing in the assessment logs because no one has bothered to update the bot in the last 19 months. Imzadi 1979→08:55, 23 July 2015 (UTC)
Double counting?
There are now more than 5 million articles in the overall stats, approx. 100k more than in the Main Page counter (4909k at the moment), so is there some double counting going on? GregorB (talk) 08:47, 5 July 2015 (UTC)
Could be, but I have the impression that, given the average number of WikiProject banners per article (1.5 or even more, perhaps?), this would result in an even higher count. GregorB (talk) 17:16, 15 July 2015 (UTC)
The bot is designed to only count an article once - otherwise we'd have about 20 million "articles" by now! The answer, I think, is simple; WikiProjects now use the bot to count many things that wouldn't count as articles such as drafts or redirects - take a look here, for example. I'm not sure why they need to do that, but I'm sure they have good reasons! We should perhaps discuss whether or not we want to exclude these types of things from the total; maybe it's not important enough to worry about, as long as people understand what that "unassessed" group contains. Walkerma (talk) 13:17, 17 July 2015 (UTC)
That's probably it. Still, these pages are not really "unassessed" then, should perhaps be displayed as "Other". GregorB (talk) 18:28, 17 July 2015 (UTC)
Walkerma and GregorB: thank you, between you, you have just answered a question that's been puzzling me – working on WikiProject Albums, I always wondered why there were literally thousands of album articles with an "NA-class" rating... it's because many of them were created by one editor and then AfD'd by another as being not notable... clicking on the album link in the list takes you to the redirected page, usually the article page for the artist. It does mean however, that it appears there are thousands and thousands of album articles that need to be improved upon, when really they don't "exist". Richard3120 (talk) 20:09, 25 July 2015 (UTC)
Richard3120, GregorB: Some projects reclassify such pages using Template:Redirect-Class, which generates a category based in Category:Redirect-Class articles. This is a "non-standard" class but it's well-used. Alternatives would be just deleting the banners, or ignoring them and accepting & making it clear that NA includes some of these redirects. The ideal, of course, would be for the bot to recognize, but we don't have an active coder at the moment for the bot. Walkerma (talk) 12:59, 27 July 2015 (UTC)
Walkerma, thanks for your reply – what you say about the Redirect-Class category is exactly what somebody told me when I posted my query on the Help:Redirect page. I'll see if there is some way around this issue, if not, we just have to live with what the bot tells us. Richard3120 (talk) 18:58, 27 July 2015 (UTC)
Mathematics B+ rating
In Wikipedia:WikiProject Mathematics/Wikipedia 1.0 Mathematics articles can get a B+ rating, I think this rating is a lower rating than the GA rating but the but WP bot 1.0 (link <--> User:WP 1.0 bot/Tables/Custom/Mathematics-Overall -->
Right now, the WP1.0 tools treat draft articles as "Other" in the WikiProject assessment tables and Tool Labs interfaces. It should list them as "Draft" and provide appropriate links to them (right now they link to non-existent article pages). Kaldari (talk) 00:56, 4 August 2015 (UTC)
Pie chart
Graphs are unavailable due to technical issues. There is more info on Phabricator and on MediaWiki.org.
I had a go, and got it looking more reasonable (to me, at least!) on the main stats page. Please check that I didn't break any code - I'm a bit of an amateur at editing things like this. Walkerma (talk) 04:37, 7 September 2015 (UTC)
Assessment logs
When looking for the assessment history/logs, it does not return any results. Entering a project results in "Project 'X' is not in the database." and entering a project name results in an empty list. I have seen in the archives that this problem has existed before. If it is just temporary, sorry for posting this. Kerenskij86 (talk) 16:25, 25 August 2015 (UTC)
The bot has run the log page several times now for the above project, but I'm not seeing a statistics page created yet. Any ideas what might be happening? John Carter (talk) 15:22, 8 September 2015 (UTC)
Hey. I've been doing assessments for Wikipedia:WikiProject United States courts and judges and, while doing so, noticed that there was no table for the project. I tried setting up the table through the guidelines, but nothing's been popping up. Everything's in order so I don't know why I can't get the bot to make any logs, tables, or anything. Wizardman00:04, 11 September 2015 (UTC)
In the last 20 minutes I've been receiving invalid certificate problems. The "https://tools.wmflabs.org/ etc" for the "Wikipedia Release Version Tools" comes up as invalid certificate. I realize I haven't put in the full URL, but you will understand what I mean. Adamdaley (talk) 01:26, 15 September 2015 (UTC)
If a project doesn't have Draft class enabled properly, you get some funny behavior with drafts that have the project banner on their talk page. That is what is going here, see Draft:Toy Story 4 and Draft:Frozen 2; the respective redirects are showing up where they're supposed to. Plantdrew (talk) 22:00, 1 December 2015 (UTC)
Hello, I'm part of the Wikiproject above and joined recently. I've noticed we don't have a quality statistic box like this and I was wondering if anyone would like to show me how to insert (and create) one to put into the Wikiproject, thank you in advance! Adog104Talk to me00:10, 3 December 2015 (UTC)
The Tampa Bay articles by quality and importance table is being generated daily (more or less), so it's just a matter of adding it to the WikiProject Tampa Bay page. The text to insert is: ((Wikipedia:Version 1.0 Editorial Team/Tampa Bay articles by quality statistics)) --Worldbruce (talk) 00:28, 3 December 2015 (UTC)
See Template talk:AbQ Pie please, the diagram could need automated updates for the total number of articles of various ratings, otherwise the diagram produces highly misleading results. Some central value to store those numbers in a template-accessible way would work. --mfb (talk) 13:41, 12 December 2015 (UTC)
Assessment Logs
Hello, I noticed the Assessment Logs tool does not return anything for any search. For example, when I enter "Africa" for project name, it says "Africa is not in the database". Also, when I click on the "log" link of any project on the project index list. For example https://tools.wmflabs.org/enwp10/cgi-bin/log.fcgi?project=Africa, I get the same message about the project not being in the database. Is this a bug, or am I doing something wrong? — Preceding unsigned comment added by Damao98 (talk • contribs) 21:36, 14 December 2015 (UTC)
Articles in draft namespace marked as Draft Class should have the draft namespace (i.e. "Draft:") in the generated links in a table like this one. They don't, so the link goes to a usually non-existent article in mainspace. (Ignore Danny Johnson (Guitar Hero) because right now this is in mainspace but marked as "draft".) Peter coxhead (talk) 21:37, 13 January 2016 (UTC)
Yes, I believe that's a known problem, but we haven't had anyone developing the code who could fix it. I was talking with some developers today, so we may be able to get some help, but I don't think we'll be able to fix it any time soon - sorry! Walkerma (talk) 16:36, 30 January 2016 (UTC)
OK, thanks for letting us know about that! We're hoping to recruit someone soon to work on developing the code, and we'll ask them to look at this - though we have some major features that need to be worked on first, so it'll take a while. Thanks, Walkerma (talk) 16:39, 30 January 2016 (UTC)
@Panintelize, I don't understand. Can you show me the link you're referring to? We have people beginning to revise the code right now, so now is a good time to get such problems fixed. Thanks, Walkerma (talk) 05:39, 17 February 2016 (UTC)
Thank you! I'll see if we can get that updated. In the meantime, I think the links given top-right (in the template) on the main WP:1 page should all be live, and we also now (as of last week) have some github repositories:
The bot seems to have stopped updating the WikiProject pages the last week or so... It has been displaying an error for me which has been "Error: No such file or directory". Adamdaley (talk) 10:47, 9 February 2016 (UTC)
I've emailed the person who does maintenance - they look as if they haven't been on recently, which is why it apparently wasn't followed up on sooner. We do have some new people starting to look at the code with a view to developing it further, though I don't think they've edited the actual bot itself; I'll make sure that wasn't the cause. Walkerma (talk) 16:39, 9 February 2016 (UTC)
Hi, this page is not edited by the wp1bot, can you please be more specific? A diff between good and bas revision would be really helpful. Kelson (talk) 17:34, 2 March 2016 (UTC)
So, if I understand correctly your question, this is "why this page/template has not been updated for the past 5 years?" Kelson (talk) 21:08, 2 March 2016 (UTC)
WikiProject Hampshire County, West Virginia
In the WikiProject Hampshire County, West Virginia assessment table, the featured articles aren't being included? Is there a fix for this? Thanks! -- West Virginian (talk)00:36, 17 March 2016 (UTC)
Sort by quality date
I'm trying to automatically generate a list of articles in a given WikiProject sorted by the date when they were reassessed as GA. This comes pretty close, but I don't see how I can sort by date. Any ideas? czar18:08, 21 March 2016 (UTC)
I've wanted to sort by date myself at times. I don't think it's possible within Wikipedia at present, but it would be a nice feature. You can copy-paste the table into a spreadsheet and then sort by the date column as a work around. You'll have to copy-paste twice since results per page can't go over 1000 (but do set it higher than 200 so you don't have to copy-paste 6 times). Plantdrew (talk) 18:17, 21 March 2016 (UTC)
WikiWork factors for Madonna project
Hi all I have seen that the Wikiwork factors for the Madonna, the work factor is stuck at 3.06 from a long long time, even though around 7-8 GAs have been promoted for the project and updated in the table. Can anyone guide me if there's something wrong? —IB[ Poke ]09:47, 16 March 2016 (UTC)
Most WikiProjects have a corresponding quality log page. However, there is no quality log page for Wikipedia:WikiProject Disambiguation (Wikipedia:Version 1.0 Editorial Team/WikiProject Disambiguation articles by quality log and Wikipedia:Version 1.0 Editorial Team/Disambiguation articles by quality log are redlinks). GeoffreyT2000 (talk) 15:36, 28 March 2016 (UTC)
Is it just me or does the Wikipedia data tool not display draftspace pages correctly? It seems to be dropping the draft namespace and just linking to the mainspace versions? For example, this listing doesn't link to a single page in draftspace but only the mainspace ones. -- Ricky81682 (talk) 19:33, 12 April 2016 (UTC)`
That's been a known problem for years. It seems that some progress has been made in the last few months; when you click on the listings, there is now a banner that provide a link to the draft. Plantdrew (talk) 21:07, 12 April 2016 (UTC)
Ok. Some projects were moving all their drafts around thinking they weren't showing up. It seems odd that one namespace has this issue and no others. -- Ricky81682 (talk) 18:27, 13 April 2016 (UTC)
It's just that the Draft namespace is far newer than any of the others. By the time it was created, or at least by the time anyone started associating drafts with projects, maintenance and enhancement of the tool had long since petered out. Perhaps the tool could have been written to work with any namespace that might be created, but plainly it wasn't. --Worldbruce (talk) 19:49, 13 April 2016 (UTC)
the problem seemed to me to be reminiscent of previous instances where multiple templates were on a single talk page, and sure enough, Education Program talk:Louisiana State University/CHEM 4558: Mass Spectrometry (Spring 2015) has lots of talk pages transcluded onto the one page. I would suggest that either Rd Education Program talk domain is excluded from the bot runs, or the program finds another way to link to articles without transcluding talk pages that include the WikiProject templates. The-Pope (talk) 02:56, 17 April 2016 (UTC)
A further Wales query. The Wikiwork factor this morning, using the live table, shows wikiwork values of ω=61,925 and Ω = 5.34 ... so do all of the historic tables for Wales going back to 8 Feb. I can just about cope with the Ω staying the same (whatever we do, it's fractal in nature, and we end up in the same place) but I'd have anticipted the ω value would increase as the number of articles increases. Whatever, I'm this morning looking askance at invariate ω & Ω and thought to ask here for comment. thanks --Tagishsimon(talk)07:42, 3 May 2016 (UTC)
I was thinking about creating some visualization of WikiProjects with requests similar to this: User:WP_1.0_bot/Tables/Project/Genetics but don't know how to get these results as json or xml.
Theo is the one in charge of this, but apparently he's not responding. I'm looking into this myself, so I'll let you know if I can find anything out. Walkerma (talk) 03:58, 8 July 2016 (UTC)
Hello @Gonzo fan2007:, both editors have been rarely editing recently. As WP:WikiProject Germany has Draft-settings active, I'll try to give you a few pointers for now. "Draft" is no standard class, but can be configured as additional "extra" class (= quality rating). You can find a somewhat technical description of the necessary parameters at Template:ReleaseVersionParameters#Extra_quality_and_importance_ratings (the example uses an extra "BPlus" class, but the same principle applies for an extra "Draft" class). If you need a practical example: the source text of Category:Germany articles by quality includes the currently active "extra1=" parameters for Germany-related drafts. An analogous setting in your project's quality category with your project's name should work. Note: Links to drafts within the bot's own article listing don't work (like here for example), but that behaviour is a known bug for drafts. General counting, statistics and categorization are OK though. GermanJoe (talk) 19:52, 13 July 2016 (UTC)
Here's my question: recently (like, within the past few months), the "recent changes" as it appears on the WikiProject page has stopped lumping together all articles' edits under a drop-down arrow. Now, at Special:RecentChangesLinked/Wikipedia:Version 1.0 Editorial Team/Star Trek articles by quality log, they show as collapsed, but when that makes it to the WikiProject page, that feature isn't working. Can you possibly point me in the right direction to find someone who can help me with this coding? — fourthords | =Λ= |02:54, 21 July 2016 (UTC)
I am in the process of setting up a new project (Wikipedia:WikiProject Wildfire) which is a spin off of another project. I wanted to get it added to this awesome bot but seem to be having some trouble. I got all the categories made, but when I run the update via here it doesn't seem to add it to the index. Not sure what I am doing wrong. --Zackmann08 (Talk to me/What I been doing) 04:26, 2 August 2016 (UTC)
Bot doesn't seem to run
Hi, I'm here representing WP:RRTF. Our article classifications chart hasn't been updated since January this year, and using the "run bot right away" tool on the bot's page doesn't seem to do anything either. I've tried making null edits to the template page and our page as well; no luck. Just wanted to let you all know, in case the issue hasn't been noticed (or is just happening to us). It's not critical for our task force to have this up and running, but if the bot is truly down, I might take down our chart as it paints a misleading picture of our article status. Thanks for reading! -- 2ReinreB2 (talk) 00:46, 2 September 2016 (UTC)
@Worldbruce -- Yeah, it looks much better now. Thanks! Somebody came along and helped with that bit of the name change; just sort of assumed they took care of everything. Thanks again, and happy editing! -- 2ReinreB2 (talk) 02:05, 4 September 2016 (UTC)
Wikiwork factor numbers
Does anyone know if the Wikiwork numbers are working anymore? In the Madonna wikiproject, it does not go down when articles are promoted, has been constant at 3.06 for the past 1 year although close to 15 articles have been promoted. —IB[ Poke ]08:40, 27 July 2016 (UTC)
I didn't think to send him an email; I just saw that the bot still seems to be doing other tasks and the owner hasn't edited in quite a while. Axem Titanium (talk) 15:31, 19 September 2016 (UTC)