User talk:Citation bot/Archive 16

This is an archive of past discussions with User:Citation bot. Do not edit the contents of this page. If you wish to start a new discussion or revive an old one, please do so on the current talk page.

Archive 10

←

Archive 14

→

Caps: Latina/O --> Latina/o

Status: {{fixed}}
Reported by: Headbomb {t · c · p · b} 18:09, 5 June 2019 (UTC)

What should happen: [1]
We can't proceed until: Feedback from maintainers

Don't de-italicize chapters

Status: {{fixed}}
Reported by: Headbomb {t · c · p · b} 05:07, 6 June 2019 (UTC)

What happens: [2]
What should happen: [3]
We can't proceed until: Feedback from maintainers

https://github.com/ms609/citation-bot/pull/1722 AManWithNoPlan (talk) 13:56, 6 June 2019 (UTC)

Link broken

Status: new bug
Reported by: Uninvited Company on behalf of User:Fram

What happens: After bot was unblocked by UninvitedCompany, the third edit I looked at caused a working link [4] to no longer work by changing it to [5] instead
Relevant diffs/links: [6].
We can't proceed until: Feedback from maintainers

{{notabug}} Neither link worked, then they both worked. academia.edu was having trouble. Thank you for reporting it though. As I go through EVERY page that has blocked the bot, I am finding a few bugs that have NEVER been reported. People just blocked the bot and reverted the edits and went on their merry way. AManWithNoPlan (talk) 13:48, 6 June 2019 (UTC)

AdsAbs database error

Status: {{fixed}} for now
Reported by: Redalert2fan (talk) 07:13, 6 June 2019 (UTC)

What happens: Checking AdsAbs database --> ! Error 999 in query_adsabs: Too many requests
We can't proceed until: Feedback from maintainers

Hello, I just tried to run the bot, however on the progress page for every AdsAbs database check the above was returned, following the api querry link I get: { "error": "Unauthorized" } . --Redalert2fan (talk) 07:13, 6 June 2019 (UTC)

Yes, the bot's usage has spiked and we used up all our authorizations. I will see about adding more queries to our limit. AManWithNoPlan (talk) 13:49, 6 June 2019 (UTC)

Citebook used for DVD on Amazon

Status: {{fixed}}
Reported by: Redalert2fan (talk) 14:13, 6 June 2019 (UTC)

What happens: [7]
We can't proceed until: Feedback from maintainers

I'm not sure what is going on here, but it seems like its taking the Amazon Standard Identification Number (ASIN): 6317859612 that is listed on the page and seeing it as an ISBN, converting cite web into cite book. In any case no books here, since its a page to buy a DVD set. Redalert2fan (talk) 14:14, 6 June 2019 (UTC) log from the web process:

  > Remedial work to prepare citations 

  - Dropping parameter "publisher" 

  - Dropping parameter "accessdate" 

  - Dropping parameter "url" 

  - Dropping parameter "publisher" 

  ~ Converting URL to ASIN parameter 

  ~ Converted ISBN10 to ISBN13 

  + Adding isbn: 6317859612

-- Redalert2fan (talk) 14:16, 6 June 2019 (UTC)

Amazon uses "ISBN" that start with "630" for their stuff. We blocked that. They are now using "631", so I blocked "61*". Once this is uploaded, this problem will be gone. https://github.com/ms609/citation-bot/pull/1724 AManWithNoPlan (talk) 14:33, 6 June 2019 (UTC)

@AManWithNoPlan: Just a question, should it still be converted to cite book like this [8]? Redalert2fan (talk) 20:07, 6 June 2019 (UTC)

yes it should be. Cite web requires a url, while cite book does not. Probably not the best choice, but some asin are books, some are dvds, and some are bottles of booze. AManWithNoPlan (talk) 20:26, 6 June 2019 (UTC)

Is it down?

@Kaldari:, I believe it needs kicked. AManWithNoPlan (talk) 19:37, 6 June 2019 (UTC)

@AManWithNoPlan: Try it now. Kaldari (talk) 19:52, 6 June 2019 (UTC)

{{fixed}} AManWithNoPlan (talk) 20:04, 6 June 2019 (UTC)

CAPS journal: Research Notes of the Aas --> Research Notes of the AAS

Status: {{fixed}}
Reported by: Redalert2fan (talk) 22:02, 6 June 2019 (UTC)

What happens: journal=Research Notes of the Aas
What should happen: journal=Research Notes of the AAS
Relevant diffs/links: [9][10]
We can't proceed until: Feedback from maintainers

https://github.com/ms609/citation-bot/pull/1728 AManWithNoPlan (talk) 22:17, 6 June 2019 (UTC)

caps: Agu --> AGU

Status: {{fixed}}
Reported by: Headbomb {t · c · p · b} 03:22, 7 June 2019 (UTC)

What happens: [11]
What should happen: AGU Fall Meeting Abstracts
We can't proceed until: Feedback from maintainers

If something more limited is desired, this should cover at least AGU Fall Meeting Abstracts and AGU Spring Meeting Abstracts. Headbomb {t · c · p · b} 03:25, 7 June 2019 (UTC)

Poor last/first1

Status: mostly {{fixed}}
Reported by: Redalert2fan (talk) 19:40, 5 June 2019 (UTC)

What happens: [12]
What should happen: [13]
We can't proceed until: Feedback from maintainers

Saving for later to look at. We get this:

  author": [
     [
       "Isaac Davison Social Issues Reporter, NZ Herald",
       "isaac.davison@nzherald.co.nz @isaac_davison"

AManWithNoPlan (talk) 03:32, 6 June 2019 (UTC)

This should help https://github.com/ms609/citation-bot/pull/1723 once uploaded. AManWithNoPlan (talk) 14:07, 6 June 2019 (UTC)

much better now, but not perfect. AManWithNoPlan (talk) 21:43, 6 June 2019 (UTC)

Yields “A, Isaac Davison Social Issues Reporter” now. AManWithNoPlan (talk) 21:45, 6 June 2019 (UTC)

Not optimal, but way better than before. Where does that A, come from though? Also would it be an idea to blacklist "reporter"? It could come up more often. Anyways thanks for the work so far! Redalert2fan (talk) 22:07, 6 June 2019 (UTC)

As for the “A” I am very confused too. On a phone, not a computer right now. AManWithNoPlan (talk) 22:12, 6 June 2019 (UTC)

I figured out the "A". https://github.com/ms609/citation-bot/pull/1734 AManWithNoPlan (talk) 16:50, 7 June 2019 (UTC)

Needs to run twice

Status: {{fixed}}
Reported by: Headbomb {t · c · p · b} 22:34, 6 June 2019 (UTC)

What happens: [14]+[15]
What should happen: [16]
We can't proceed until: Feedback from maintainers

https://github.com/ms609/citation-bot/pull/1738 AManWithNoPlan (talk) 20:09, 7 June 2019 (UTC)

Date off by one day

Status: {{fixed}}
Reported by: Redalert2fan (talk) 09:40, 7 June 2019 (UTC)

What happens: [17]
What should happen: 2017-08-13
Relevant diffs/links: reference
We can't proceed until: Feedback from maintainers

Article was posted on 2017-08-13 08:06 according to the page. Redalert2fan (talk) 09:40, 7 June 2019 (UTC)

Time zones. Both dates are present. Looks like it’s Zulu time in meta data and local time in the visible area. AManWithNoPlan (talk) 13:10, 7 June 2019 (UTC)

https://github.com/ms609/citation-bot/pull/1737 AManWithNoPlan (talk) 18:27, 7 June 2019 (UTC)

Bad series

Status: {{fixed}}
Reported by: —David Eppstein (talk) 20:34, 7 June 2019 (UTC)

What happens: https://en.wikipedia.org/w/index.php?title=Widest_path_problem&diff=prev&oldid=900821082
Relevant diffs/links: https://en.wikipedia.org/w/index.php?title=Widest_path_problem&diff=next&oldid=900821082
We can't proceed until: Feedback from maintainers

In some sense this is GIGO. But in general: The ACM supplies metadata saying that the "series" of its conference proceedings is the abbreviated title for the same conference — in this case, we have title "Proceedings of the 20th Annual ACM-SIAM Symposium on Discrete Algorithms" and series "SODA '09". This is an incorrect usage of the series parameter, and the reference already had it correctly incorporated directly into the title ("Proceedings of the 20th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA '09)"). This bot edit made it worse, in three ways: (1) it uncorrected the metadata by splitting out the bogus series again, (2) it left the same series name incorporated into the title parameter, creating unnecessary and redundant duplication of the same material twice, and (3) it capitalized an initialism that should correctly have been left all-uppercase. If the bot cannot have the intelligence to recognize that this is a case of GIGO and that it would be better left unchanged, it should not run at all. Do I need to re-add the bot-exclusion flag for this article that AManWithNoPlan removed immediately before making this bad edit? Maybe the best general fix would be to ignore the series parameter when it comes from ACM? —David Eppstein (talk) 20:34, 7 June 2019 (UTC)

I will do the general solution AManWithNoPlan (talk) 21:02, 7 June 2019 (UTC)

Bot unblocked

Pursuant to blocking policy and bot policy, I have unblocked the bot.

I would strongly encourage all those involved in maintaining the bot to review the nature of the bot's operation and ensure that all tasks are approved and have community support. A new BRFA would be appropriate for any tasks that have been added that are outside the initial approval. Uninvited Company 17:26, 5 June 2019 (UTC)

{{fixed}} archive flag

PMC canada

Status: {{fixed}}
Reported by: Headbomb {t · c · p · b} 02:12, 7 June 2019 (UTC)

What happens: [18]
What should happen: Same + [19]
We can't proceed until: Feedback from maintainers

EVERYONE needs their own PMC these days 🙄 AManWithNoPlan (talk) 03:24, 7 June 2019 (UTC)

Actually, the Canadian PMC site is now down, because it made no sense to have a seperate one. Headbomb {t · c · p · b} 03:29, 7 June 2019 (UTC)

I need an Oprah meme: “and you get a PMC website...". AManWithNoPlan (talk) 04:11, 7 June 2019 (UTC)

https://github.com/ms609/citation-bot/pull/1732 AManWithNoPlan (talk) 15:50, 7 June 2019 (UTC)

citeseerx added or changed

Hello, this edit: diff added a citeseerx according to the edit summary, however I thought this function was removed to unblock the bot. Now I'm not familiar with this so could you please take a look? Thanks, Redalert2fan (talk) 22:54, 7 June 2019 (UTC)

the citeseerx url was converted. This is an upgrade since linking to the citaweex page is better than the PDF directly since it is less questionable from a copyright perspective and most of the time you just want to see the abstract AManWithNoPlan (talk) 22:58, 7 June 2019 (UTC)

Understood, thanks for the explanation. Redalert2fan (talk) 23:05, 7 June 2019 (UTC)

I have often criticized the addition of citeseerx links, but I agree that when they are already there it is better to convert them in this way. —David Eppstein (talk) 01:05, 8 June 2019 (UTC)

flag for archive {{notabug}}

More caps

Status: {{fixed}}
Reported by: Redalert2fan (talk) 23:02, 7 June 2019 (UTC)

What happens: Aps Division of Nuclear Physics Meeting Abstracts
What should happen: APS Division of Nuclear Physics Meeting Abstracts
Relevant diffs/links: edit that was made - adsabs with booktitle = {APS Division of Nuclear Physics Meeting Abstracts},
We can't proceed until: Feedback from maintainers

ISBN with spaces only

Need to fix these. AManWithNoPlan (talk) 20:17, 7 June 2019 (UTC)

https://github.com/ms609/citation-bot/pull/1742 AManWithNoPlan (talk) 03:53, 9 June 2019 (UTC)

Caps: Its

Status: {{fixed}}
Reported by: Headbomb {t · c · p · b} 16:14, 8 June 2019 (UTC)

What happens: [Mechanized Mathematics and Its Applications]
What should happen: Leave 'its', 'Its' or 'ITS' alone.
We can't proceed until: Feedback from maintainers

url to chapter-url when chapter has wikilink

Status: {{fixed}}
Reported by: Trappist the monk (talk) 15:49, 10 June 2019 (UTC)

What happens: changed |url= to |chapter-url= when |chapter= has a wikilink (diff)
We can't proceed until: Feedback from maintainers

bot changed this:

{{cite book |chapter= [[Bangla Bhasha Procholon Ain, 1987]]|script-chapter=বাংলা ভাষা প্রচলন আইন, ১৯৮৭ |trans-chapter=Bengali Language Implementation Act, 1987 |chapter-format= |title= Bangladesh Code|script-title=বাংলাদেশ কোড |trans-title= |url= http://bdlaws.minlaw.gov.bd/pdf/705___.pdf|dead-url= |language=bn |volume=27 |issue= |others= |edition=Online |location=Dhaka |publisher=Ministry of Law, Justice and Parliamentary Affairs, Bangladesh |publication-date= |page= |pages= |archive-url= |archive-date= |access-date=2019-05-23 }}

"Bangla Bhasha Procholon Ain, 1987" বাংলা ভাষা প্রচলন আইন, ১৯৮৭ [Bengali Language Implementation Act, 1987]. Bangladesh Code বাংলাদেশ কোড (PDF) (in Bengali). Vol. 27 (Online ed.). Dhaka: Ministry of Law, Justice and Parliamentary Affairs, Bangladesh. Retrieved 2019-05-23. {{cite book}}: Cite has empty unknown parameter: |dead-url= (help); Invalid |script-chapter=: missing prefix (help); Invalid |script-title=: missing prefix (help)

to this:

{{cite book |chapter= [[Bangla Bhasha Procholon Ain, 1987]]|script-chapter=বাংলা ভাষা প্রচলন আইন, ১৯৮৭ |trans-chapter=Bengali Language Implementation Act, 1987 |title= Bangladesh Code|script-title=বাংলাদেশ কোড |trans-title= |chapter-url= http://bdlaws.minlaw.gov.bd/pdf/705___.pdf|dead-url= |language=bn |volume=27 |issue= |others= |edition=Online |location=Dhaka |publisher=Ministry of Law, Justice and Parliamentary Affairs, Bangladesh |date= |page= |pages= |archive-url= |archive-date= |access-date=2019-05-23 }}

"[[Bangla Bhasha Procholon Ain, 1987]]" বাংলা ভাষা প্রচলন আইন, ১৯৮৭ [Bengali Language Implementation Act, 1987] (PDF). Bangladesh Code বাংলাদেশ কোড (in Bengali). Vol. 27 (Online ed.). Dhaka: Ministry of Law, Justice and Parliamentary Affairs, Bangladesh. Retrieved 2019-05-23. {{cite book}}: Cite has empty unknown parameter: |dead-url= (help); Invalid |script-chapter=: missing prefix (help); Invalid |script-title=: missing prefix (help); URL–wikilink conflict (help)

—Trappist the monk (talk) 15:49, 10 June 2019 (UTC)

The fix for the above is to move the wikilink to |trans-chapter= because that parameter is not linked by |chapter-url=. Yeah, this is only good if there is a valid target for the English translation title.

{{cite book |chapter=Bangla Bhasha Procholon Ain, 1987 |script-chapter=বাংলা ভাষা প্রচলন আইন, ১৯৮৭ |trans-chapter=[[Bengali Language Implementation Act, 1987]] |title= Bangladesh Code|script-title=বাংলাদেশ কোড |trans-title= |chapter-url= http://bdlaws.minlaw.gov.bd/pdf/705___.pdf|dead-url= |language=bn |volume=27 |issue= |others= |edition=Online |location=Dhaka |publisher=Ministry of Law, Justice and Parliamentary Affairs, Bangladesh |date= |page= |pages= |archive-url= |archive-date= |access-date=2019-05-23 }}

"Bangla Bhasha Procholon Ain, 1987" বাংলা ভাষা প্রচলন আইন, ১৯৮৭ [Bengali Language Implementation Act, 1987] (PDF). Bangladesh Code বাংলাদেশ কোড (in Bengali). Vol. 27 (Online ed.). Dhaka: Ministry of Law, Justice and Parliamentary Affairs, Bangladesh. Retrieved 2019-05-23. {{cite book}}: Cite has empty unknown parameter: |dead-url= (help); Invalid |script-chapter=: missing prefix (help); Invalid |script-title=: missing prefix (help)

—Trappist the monk (talk) 15:56, 10 June 2019 (UTC)

Suggest BRFA

I believe there are two problems at work here:

There is legitimate concern that this bot's operator of record is, at present, not taking a sufficient, active interest in the bot's maintenance and operation. Others, notably AManWithNoPlan, have since taken up that mantle, and the approval should perhaps be updated to reflect that.
There is a question whether the bot meets current policy regarding attribution of edits. I believe this is a judgment call that would best be addressed by members of the BAG.

Accordingly, I believe the best way forward would be to submit a new request for approval at WP:BRFA. @AManWithNoPlan:, if you wish to step up and be the maintainer of record, it would fall to you to make the formal request. It seems to me that an accepted BRFA would make it clear that the bot should be unblocked. Uninvited Company 17:48, 29 May 2019 (UTC)

The concerns in the first bullet may be 'legitimate', but it is also unfounded and irrelevant. Bot policy require that concerns and messages are addressed by either the bot op or others maintainers and helpers. Since maintainers are clearly active, the operation of the bot is fully compliant with WP:BOTCOMM. Likewise, WP:BOTPOL requires that the operator of the bot is identified, and they are: User:Smith609, as the banner at the top of User:Citation bot makes it clear. Likewise, User talk:Citation bot makes it clear who the maintainers are: User:Smith609, User:Kaldari, and User:AManWithNoPlan. Headbomb {t · c · p · b} 21:50, 29 May 2019 (UTC)

I do understand that point of view and am sympathetic to it. On the other hand, it is not clear to me that the maintainers and helpers are speaking with one voice. Smith609 has stated that the bot is autonomous and responsible for its own edits. On the other hand, AManWithNoPlan has implied here and stated elsewhere that the bot is user-directed (a point of view you appear to share, based on the fact that you warn users of the bot that they are responsible for the bot's edits when running it on a large number of articles). Bot policy requires that the bot be one or the other, and specifies expectations of the bot and the responsibilities of the operator accordingly. It appears to me that this is the core of the concerns Xover has expressed and which others share. Uninvited Company 23:14, 29 May 2019 (UTC)

The Bot takes responsibility for its actions. Warning people to be observant is not a removal of the Bot’s responsibility but an attempt to make people feel obligated to check up on the bot. Until my recent (2019) pull that detected template errors there could be horrible GIGO problems. Things are much better now. AManWithNoPlan (talk) 00:03, 30 May 2019 (UTC)

This was intended to be a warning against running then very new code on a large number of articles, while the state of the bot was very much in flux (e.g. issues with unthrolled rates). It did not absolve maintainers of their responsibilities. Headbomb {t · c · p · b} 03:03, 30 May 2019 (UTC)

Changing {{Cite web}} to {{Cite news}} when Cite web is correct template

Status: new bug
Reported by: Pyxis Solitary yak 05:30, 13 June 2019 (UTC)

What should happen: bot needs to recognize changes that took effect 29 July 2016 between use of {{Cite web}} vs. {{Cite news}}; in which {{Cite news}} is now used for "offline (paper) sources", because "given the same set of valid parameters" the output between the two "is exactly the same". See Template:Cite news > Choosing between {{Cite web}} and {{Cite news}}.
Relevant diffs/links: [20]: The Daily Beast and The Daily Telegraph citations changed to News, including parameters titles: TDBeast source is website-only (no print issue); TDTelegraph source is a webpage.
We can't proceed until: Feedback from maintainers

The literal first line of Template:Cite news is This Citation Style 1 template is used to create citations for news articles in print, video, audio or web. --Izno (talk) 12:19, 13 June 2019 (UTC)

Which is modified by the addition of section "Choosing between {{Cite web}} and {{Cite news}}" and subsequent instructions: "Before 2014, editors needed to decide whether to use {{Cite web}} or {{Cite news}} based on their features. In 2014, however, most of the differences between the two templates were eliminated. As of 29 July 2016, {{Cite web}} and {{Cite news}} have the following differences: {{Cite news}} can be used for offline (paper) sources whereas {{Cite web}} generates a missing URL error when no URL is provided...But given the same set of valid parameters, their output is exactly the same". "Offline (paper) sources" is distinct. Pyxis Solitary yak 12:45, 13 June 2019 (UTC)

The discussion since the RFC regarding the use of one template vice the other usually tends toward "use cite web only where necessary" to cite a web resource which does not fall under any of the other CS1 templates. Feel free to peruse the Help talk:CS1 archives and/or the WP:Citing sources archives. Just because the display is the same now in many/most cases doesn't mean it is not better to use cite news rather than cite web, as cite news indicates intent better. Now, you might reasonably say that the bot shouldn't make an edit where that is the only kind of edit (i.e., the switch is cosmetic), but that's a separate concern to "one is correct and one is not". --Izno (talk) 14:09, 13 June 2019 (UTC)

Honestly, there isn't enough time in life to "peruse" a ton of Wikipedia discussion archives. It's okay if you can't pin-point the one that directly relates to this issue, but telling editors to dumpster-dive archives isn't helpful. Using {{Cite news}} for "Offline (paper)" sources is not sinking in. What does "offline" sources mean in relation to citation templates? Simply put: for a news-associated item/article/opinion/report/column that is not accessible on the web and only available the 'old school' way, you not only provide author-title-publication-publisher-date ... you also include volume-issue-page. But if a news-associated source, such as a report or opinion piece, was published on the web you use {{Cite web}} -- because the source being used is a webpage on a website. A cosmetic change is precisely what happens when a citation template is changed from {{Cite web}} to {{Cite news}} when the former provides the exact same content and output. Pyxis Solitary yak 03:58, 14 June 2019 (UTC)

What's not helpful is stubbornly asserting that you are correct about the general gist of things without doing the research or having been in the discussions in question, especially when there is template documentation directly contradicting your opinion. I won't respond here further. --Izno (talk) 04:12, 14 June 2019 (UTC)

As a generic observation: the bot does edits that change things for the readers and the editors benefits. For example removing extra spaces doesn’t change display but makes text match what is seen. AManWithNoPlan (talk) 15:06, 15 June 2019 (UTC)

{{notabug}} I trust the dail beast to know itself well: The Daily Beast is dedicated to independent journalism, pursued without fear or favor. AManWithNoPlan (talk) 16:33, 16 June 2019 (UTC)

Do not remove nobots

{{resolved}} I don't care anymore if {{bots|deny=Citation bot}} is a "blunt instrument". Do not remove it from any articles until the problems with the bot has have been fixed. I am out of patience with bad changes. Thank you. Hawkeye7 (discuss) 07:50, 15 June 2019 (UTC)

Where has this been removed that the bot wasn't 'fixed'? Headbomb {t · c · p · b} 12:03, 15 June 2019 (UTC)

lots of those were added over five years ago. Secondly, I have fixed literally dozens of gigo bugs looking at these pages. Lastly, I have fixed about a dozen issues dealing with all sorts of citation template misuse and the bots understandable confusion looking at these pages. For example, the ISBN to ISBN13 conversion is now only done if the book was issued 2007 or newer — which means it has an ISBN13. AManWithNoPlan (talk) 13:04, 15 June 2019 (UTC)

@AManWithNoPlan: I'd rather see the problematic bot edits before even speculating about what's going on. For all we know, there's vandalism at play, or some random person removing nobots without understanding what it is they are doing. I highly doubt the bot is going around removing {{nobots}}. Headbomb {t · c · p · b} 18:42, 15 June 2019 (UTC)

I doubt the bot is, but I have definitely seen AManWithNoPlan remove Citation bot specific exclusion tags. In all the cases I saw, it was because the problematic behavior had indeed been fixed. —David Eppstein (talk) 18:45, 15 June 2019 (UTC)

The bot exclusion tag was manually removed from John Glenn [21], but the problematic behaviour has not yet been fixed, leading to it recurring when someone - for no reason that I can discern - chose to run the bot against the article. [22] Hawkeye7 (discuss) 23:33, 15 June 2019 (UTC)

Seems something like you need to tell JJMC89 more than here. Headbomb {t · c · p · b} 23:39, 15 June 2019 (UTC)

also a good opportunity to block the specific citation that is problematic. AManWithNoPlan (talk) 03:04, 16 June 2019 (UTC)

I fixed the citations that caused trouble. The issues are 1. Meta data for non journals being presented to the bot using journal style descriptions. The bots additions add information, but poorly allocated to the template parameters. 2. Local vs Zulu time. Can give you a date that is one different than expected. Really unsolvable. AManWithNoPlan (talk) 15:45, 16 June 2019 (UTC)

Error authenticating. Resetting. Please try again.

Status: {{notabug}}
Reported by: — Chris Capoccia 💬 14:11, 15 June 2019 (UTC)

What happens: Trying to run Citation bot on a page with commit unchecked from https://tools.wmflabs.org/citations/ and all you get is "Error authenticating. Resetting. Please try again."
We can't proceed until: Feedback from maintainers

The bot is currently blocked. See above. In the mean time use gadget mode. AManWithNoPlan (talk) 15:00, 15 June 2019 (UTC)

I get the same error with the gadget. Is there something I'm doing wrong? –Fredddie ™ 17:07, 15 June 2019 (UTC)

I doubt that. The gadget is when you are in an edit window and click the citation button next to the preview and save buttons. What I meant by “gadget” was not very clear-and might still not be. AManWithNoPlan (talk) 17:56, 15 June 2019 (UTC)

OK thanks for the clarification; that seems to be working. I have an Expand citations link on the sidebar and that's what's not working. –Fredddie ™ 18:04, 15 June 2019 (UTC)

The sidebar link is triggering the bot itself. The citation expander gadget that works is the citations button near the bottom of the edit window when you are in edit mode. Headbomb {t · c · p · b} 18:22, 15 June 2019 (UTC)

because of other bugs, the gadget only expands DOIs for journal articles and not DOIs for book chapters or PMIDs. Before this weekend, I could run Citation bot on a whole page with the box unchecked for commit edits, and then copy/paste the output into the page. — Chris Capoccia 💬 19:01, 15 June 2019 (UTC)

There is NO difference between gadget and non-gadget for quite some time. AManWithNoPlan (talk) 19:28, 15 June 2019 (UTC)

ok, now that i've read WP:UCB, it's clear that i had no idea what "gadget" was. I thought the thing on the top of the edit window that lets you insert a cite web/news/book/journal was the gadget. lol! i don't know what that thing is called, but it's more limited. — Chris Capoccia 💬 19:48, 15 June 2019 (UTC)

@Chris Capoccia: well to be fair, that is also part of a gadget (WP:REFTOOLBAR), just not this one. I'll clarify the notice at the top of User:Citation bot. Headbomb {t · c · p · b} 19:53, 15 June 2019 (UTC)

@Chris Capoccia: see the new hatnote at User:Citation bot and updated instructions at Wikipedia:Citation expander. Does that make things clearer? Headbomb {t · c · p · b} 20:57, 15 June 2019 (UTC)

yes, it sends people in the right direction to figure things out. — Chris Capoccia 💬 22:55, 15 June 2019 (UTC)

We are rolling out the new OAuth based code while blocked, thus instead of failing at write, the bot fails instantly now. AManWithNoPlan (talk) 21:59, 15 June 2019 (UTC)

OK. this is makes sense about why things behaved differently now vs the previous block. — Chris Capoccia 💬 22:55, 15 June 2019 (UTC)

ScienceDirect should not be an acceptable title

Status: {{fixed}}
Reported by: (t) Josve05a (c) 20:00, 16 June 2019 (UTC)

What should happen: |title=ScienceDirect should be replaced.
Relevant diffs/links: https://en.wikipedia.org/w/index.php?title=User:Josve05a/cite-sandbox&diff=902132729&oldid=902132697
We can't proceed until: Feedback from maintainers

https://github.com/ms609/citation-bot/pull/1791 AManWithNoPlan (talk) 21:51, 16 June 2019 (UTC)

More volume/issue cleanup

Status: {{fixed}}
Reported by: (t) Josve05a (c) 20:07, 16 June 2019 (UTC)

What happens: |volume=Vol. 46 Iss. 3
What should happen: |volume=46|issue=3
Relevant diffs/links: https://en.wikipedia.org/w/index.php?title=Space_colonization&diff=902133681&oldid=902070685
We can't proceed until: Feedback from maintainers

https://github.com/ms609/citation-bot/pull/1792 AManWithNoPlan (talk) 21:51, 16 June 2019 (UTC)

Bad title

One more for the bad title list: Cur_Title --Redalert2fan (talk) 16:58, 17 June 2019 (UTC)

Diff? Headbomb {t · c · p · b} 17:25, 17 June 2019 (UTC)

I didn't save it because it was a bad edit. However it happens on https://sangeetnatak.gov.in/sna/citation_popup.php?id=XXXX&at=5 where the id is any between 1436 and 1721, any other id's give a blank page (though with the same title). It seems like these are short biographies of people and the actual title should be their name which is on the page in bold. The actual title displayed in my web browser seems to be CUR_TITLE (all caps) Redalert2fan (talk) 17:42, 17 June 2019 (UTC)

new handle resolver

Status: {{fixed}}
Reported by: Headbomb {t · c · p · b} 20:38, 17 June 2019 (UTC)

What should happen: [23]
We can't proceed until: Feedback from maintainers

Title: WordPress › Error

Status: {{fixed}}
Reported by: Redalert2fan (talk) 21:13, 17 June 2019 (UTC)

What happens: Title: " WordPress › Error " is added
Relevant diffs/links: [24]
We can't proceed until: Feedback from maintainers

Seems to be because the page is dead, but not a good title anyways. Redalert2fan (talk) 21:13, 17 June 2019 (UTC)

I marked the link as dead, but "Archived Copy" would still be changed to "WordPress › Error". Redalert2fan (talk) 21:32, 17 June 2019 (UTC)

Reblocked

Unfortunately, the bot is being used by someone who is evading a block to continually follow around two other editors; and there is apparently no way to prevent this with the current configuration. And nothing to prevent other LTAs from doing the same to other editors. If the bot can be modified to resolve this problem, an admin can unblock without talking to me first, but my understanding is this is going to be very difficult. I'm sorry, I know this bot is very useful to a lot of good faith volunteers.

For background, see here. --Floquenbeam (talk) 23:11, 13 June 2019 (UTC)

Yeah, we can't have it being used to harass other editors. Smith609 AManWithNoPlan pinging you both to make you aware.-- 5 albert square (talk) 23:21, 13 June 2019 (UTC)

I am curious how the bot could be weaponized in such a way, but perhaps explaining that might just give others bad ideas. I generally use the gadget mode myself. AManWithNoPlan (talk) 23:28, 13 June 2019 (UTC)

I'll send you an email explaining. --Floquenbeam (talk) 23:34, 13 June 2019 (UTC)

There are some ways around this, but sadly there are lots of WP:BEANS at play, and could not garantee that a technically competent person wouldn't be allowed to trigger the bot. Looks like WP:OAuth or similar will be needed, since we've got people abusing the bot. What a sad day. Headbomb {t · c · p · b} 23:38, 13 June 2019 (UTC)

I'm told there's a gadget that still works, which makes me feel slightly less guilty. but yeah, sorry, don't see a way around a block for now. --Floquenbeam (talk) 23:42, 13 June 2019 (UTC)

There is, yes. It's, however, a huge pain in the ass to trigger on multiple articles. Headbomb {t · c · p · b} 23:47, 13 June 2019 (UTC)

@Kashmiri: ——SerialNumber54129 10:14, 14 June 2019 (UTC)

Floquenbeam Or someone else. Please unblock the account. 20:55, 18 June 2019 (UTC)

Is the OAuth work completed such that it is no longer possible to trigger the bot anonymously? Uninvited Company 15:35, 19 June 2019 (UTC)

yes. AManWithNoPlan (talk) 16:10, 19 June 2019 (UTC)

Will unblocking the bot resume the bot's behaviour of making fully automated edits through another editor's account? Or prevent blocked editors from activating the bot? Headbomb {t · c · p · b} 16:18, 19 June 2019 (UTC)

the bot being blocked has no effect on editing using other people’s accounts. The writing to other people’s accounts is now disabled in the bot; thus we need the bot unblocked so it can edit files itself (and naturally included verified user ids in the edit summary) AManWithNoPlan (talk) 18:25, 19 June 2019 (UTC)

Does the bot check if users requesting activation are unblocked before making edits (which is what lead to the most recent block)? Headbomb {t · c · p · b} 19:18, 19 June 2019 (UTC)

yes. AManWithNoPlan (talk) 19:27, 19 June 2019 (UTC)

Doesn't seem to be anything in the way of an unblock then. Best way to go forward is to make an WP:UNBLOCK request. Headbomb {t · c · p · b} 19:37, 19 June 2019 (UTC)

No need, I just saw this (note that yesterday's ping didn't work because there was no signature). If the bot can no longer be used by blocked accounts and/or blocked IPs, and clearly identifies who is requesting the edit, then the original reason for the block no longer applies. I'll unblock in a minute. This does not mean I understand everything about what is going on - particularly about whether the bot currently, or yesterday, is/was in violation of bot policy. So if there are still technical problems it's possible the bot might get reblocked again. But I would imagine to get the bugs out of the system you need to actually use it, so unblocking. --Floquenbeam (talk) 19:49, 19 June 2019 (UTC)

I just tried it on my sandbox page and got an error, fwiw. Uninvited Company 20:10, 19 June 2019 (UTC)

AManWithNoPlan Smith609 I'm also getting an error message, I suspect it may be the same as UninvitedCompany. I tried running the bot on the page Lothian Buses, I click to process the page and it comes up that it's going to commit edits to the page and under that it says "! API call failed: The authorization headers in your request are not valid: No approved grant was found for that authorization token". Tried running the bot on a category and got the same.-- 5 albert square (talk) 23:17, 19 June 2019 (UTC)

I never claimed it worked. AManWithNoPlan (talk) 23:47, 19 June 2019 (UTC)

closing thread since unblocked. Block {{fixed}}. Not don’t work yet. AManWithNoPlan (talk) 23:49, 19 June 2019 (UTC)

Remove postcript=

Way back when, citation bot was doing things like [25], adding |postscript= to citations that got followed by a period.

This behaviour is long gone, but the stuff remains in article. This should be cleaned up because things like

{{Cite journal | journal = Nature | volume = 389 | issue = 162–164 | year = 1997 | doi = 10.1038/38241 | title = Direct observation of a fractional charge | first1 = R. | last1 = de-Picciotto | first2 = M. | last2 = Reznikov | first3 = M. | last3 = Heiblum | first4 = V. | last4 = Umansky | first5 = G. | last5 = Bunin | first6 = D. | last6 = Mahalu | pages = 162 | postscript = }}.

results in

de-Picciotto, R.; Reznikov, M.; Heiblum, M.; Umansky, V.; Bunin, G.; Mahalu, D. (1997). "Direct observation of a fractional charge". Nature. 389 (162–164): 162. doi:10.1038/38241..

Notice the double dot at the end.

These should be cleaned up to

{{Cite journal | journal = Nature | volume = 389 | issue = 162–164 | year = 1997 | doi = 10.1038/38241 | title = Direct observation of a fractional charge | first1 = R. | last1 = de-Picciotto | first2 = M. | last2 = Reznikov | first3 = M. | last3 = Heiblum | first4 = V. | last4 = Umansky | first5 = G. | last5 = Bunin | first6 = D. | last6 = Mahalu | pages = 162 }}

to give the proper

de-Picciotto, R.; Reznikov, M.; Heiblum, M.; Umansky, V.; Bunin, G.; Mahalu, D. (1997). "Direct observation of a fractional charge". Nature. 389 (162–164): 162. doi:10.1038/38241.

So basically, if you find |postscript= (case/spacing insensitive, as long as the content of the comment is 'none'), remove it. And if a CS1 template without |mode=cs2, or a {{citation}} with |mode=cs1, is followed with a dot after the final }}, remove that dot. Headbomb {t · c · p · b} 06:37, 16 June 2019 (UTC)

{{fixed}} as well as it be. AManWithNoPlan (talk) 23:48, 19 June 2019 (UTC)

Bot down for now

Back soon we hope. In mean time use gadget mode.

Need to use user OAuth

Status: {{fixed}}
Reported by: AManWithNoPlan (talk) 21:22, 5 June 2019 (UTC)

What should happen: should log which user activates it
We can't proceed until: Feedback from maintainers

Link to in progress pull, if someone can look at it. https://github.com/ms609/citation-bot/pull/1624 and comment. AManWithNoPlan (talk) 03:20, 6 June 2019 (UTC)

It might be a good idea to not require OAuth in draft/userspace so newbies can use it without going through the extra step. Headbomb {t · c · p · b} 04:07, 6 June 2019 (UTC)

An in WP:SANDBOX! — kashmīrī ^TALK 05:21, 6 June 2019 (UTC)

PLEASE TRY this. It appears to work now. AManWithNoPlan (talk) 19:58, 17 June 2019 (UTC)

Checked, seems to be working with "Expand citations" and web interface, both single page and category. Note that when running from category the edit summary says (still) "You can use this bot yourself.", as can be seen here. If a user activates it suggest changing it to: "You can use this tool yourself." or removing that part completely when not running from the main bot account. Redalert2fan (talk) 20:13, 17 June 2019 (UTC)

Seems to work indeed. I might need further testing, but I suggest making an unblock request. Headbomb {t · c · p · b} 20:15, 17 June 2019 (UTC)

Uh, the weird thing seems to be that when I ask to run the bot, the edits are made under my account, rather than under the bot's account... Headbomb {t · c · p · b} 20:18, 17 June 2019 (UTC)

When the bot is blocked you become the bot yourself? hmmmm... We probably don't want that as a feature otherwise we'll never get unblocked haha. User:AManWithNoPlan Is this a mistake or for testing? Using the method of edit --> press citations while the bot is blocked already uses your own account so I don't see why OAuth is useful for this field. But an educated guess makes me believe you know that already. I guess we do have a cool tag now when using this method. My comment above is related to this then. --Redalert2fan (talk) 20:34, 17 June 2019 (UTC)

Can you be a bit more precise in your description to reproduce? OAuth attributes the edits to you--that's just how it works. Citation bot won't make edits any longer. From your description there it seems that this is Working As Designed. --Izno (talk) 21:00, 17 June 2019 (UTC)

Yes, sure I understand that this is how OAuth works, but it was my understanding that it would be used to verify the user activating the bot and then make edits via the bot account just as using User:InternetArchiveBot via the bot job mode on their web interface makes edits under the bot account and not your own as an example. Redalert2fan (talk) 21:07, 17 June 2019 (UTC)

(EC) What I believe most people thought would happen was that OAuth would be used to authenticate the user requesting the bot to be triggered on a page, and upon receiving verified credentials, the bot would perform the edit on behalf of that user (possibly after checking if that user was not blocked). Headbomb {t · c · p · b} 21:08, 17 June 2019 (UTC)

Running 'as you' vs 'giving you credit'

Extended content

that’s an interesting point. I think the subtle difference between ‘as you’ and ‘giving you credit’ was not obvious. Since the bot runs ‘as the user’ it’s not blockable now. But, you now get 100% credit for the edits, since they are made ‘as you’ AManWithNoPlan (talk) 21:17, 17 June 2019 (UTC)

As AMWNP says. --Izno (talk) 21:19, 17 June 2019 (UTC)

Actually, while the bot may not be blockable, 'you' are because 'you' are running an automated bot on your own account without approval. Headbomb {t · c · p · b} 21:22, 17 June 2019 (UTC)

Yeah that's why I was confused, using the category mode I am actually running the whole bot myself without approval. And who is responsibly for the edits? the bot operator or me? and If I'm responsible will the bot operator get blocked if I ignore policy/start with vandalism? Or am I the botop now? --Redalert2fan (talk) 21:28, 17 June 2019 (UTC)

The person responsible for the edits, as always, has been the person who presses the button to start the bot. This person is now also the person attributed, under the account which that person is using to log in to the bot.

If you are somehow able to make the bot vandalize in the sense we mean vandalize (possible, but unlikely), you will be blocked. Not the bot.

More likely, you will now make edits the bot would have made. If you make edits that are subpar but not outright vandalism, you are responsible for those edits and must review them whenever they are performed.

The bot operator is still Whoever It Is (out of all the recent discussion, it is still not evident to me--I suppose it is Smith still), and the bot is still approved (to do the operations it was doing in 2011 or 2012, whenever it last ran BRFA).

In essence, OAuth is just a technical implementation of the current bot policy.

We might have a meaningful discussion about what that means in the context of e.g. mass category edits. --Izno (talk) 22:15, 17 June 2019 (UTC)

Everyone should have actually read the discussions more carefully over the last year about this issue, since literally none of this is news (obviously we all made wrong assumptions, since no one said “you seem to be talking about different things”). AManWithNoPlan (talk) 21:37, 17 June 2019 (UTC)

So to get it clear the intention from you is and was to run this via the own user account? meaning the bot account wil not be used anymore? Honestly it doesn't matter to me under what name or account edits are made, but I prefer not to get blocked for running an automated bot on my own account without approval :) Redalert2fan (talk) 21:45, 17 June 2019 (UTC)

There is also the greater issue that if the bot is malfunctioning, now you possibly have to block several user accounts to prevent the damage. The bot was not authorized to do automated runs under random accounts, and likely will not be either. For instance, I just asked it to do a ~25 article run here [26]. I could have easily asked it to do a 1000+ article run, causing massive flooding of watchlist and the like. Headbomb {t · c · p · b} 22:50, 17 June 2019 (UTC)

"Citation bot" is no longer a bot; it is no longer subject to bot policy and no longer requires approval. * Pppery * _{it has begun...} 02:21, 18 June 2019 (UTC)

The edits are tagged. AManWithNoPlan (talk) 02:29, 18 June 2019 (UTC)

And that's relevant how? No Wikipedia:Bots/Requests for approval/AutoWikiBrowser needed to be submitted by the developers of AWB, even though AWB tags its edits and can be used to make large numbers of edits in one go. * Pppery * _{it has begun...} 02:33, 18 June 2019 (UTC)

it’s only tangentially relevant; in that you can still see all the citation bot edits in a single list easily. It’s a logistical point, not an argument. AManWithNoPlan (talk) 02:37, 18 June 2019 (UTC)

"Citation bot" is no longer a bot. Completely wrong. AWB is a semi-automated process, every edit is reviewed before making it. [27] was not. This was fully automated. Headbomb {t · c · p · b} 02:39, 18 June 2019 (UTC)

Here is an admin making 39 fully-automated edits using XFDcloser. The only limit to the number of edits that can be made in one click with XFDcloser is the number of pages included in one XFD nomination, which can get into the hundreds. Yet XFDcloser is not considered a bot and has no BRFA. How is "Citation bot" different, other than using OAuth rather than being coded in Javascript, which is pure semantics that should have no effect on whether something is considered a bot? (courtesy ping: Plastikspork, Evad37) * Pppery * _{it has begun...} 03:13, 18 June 2019 (UTC)

That's a script, which does a very specific task. When the user clicks it, they know exactly what will happen. It cannot be used by random people to edit thousands of mainspace pages without review. Headbomb {t · c · p · b} 03:20, 18 June 2019 (UTC)

In any case, could the recent OAuth changes be undeployed ASAP? In the meantime, I have made EF request to prevent unauthorized bot runs on user accounts. Headbomb {t · c · p · b} 02:44, 18 June 2019 (UTC)

I don’t actually have time tonight to write code. Considering that bot was fine when random strings was used for usernames, I find suddenly assigning blame to be evil odd. We would need to add layer of user verification to the code also, since banned users can get valid tokens on meta: they just can’t edit with them . AManWithNoPlan (talk) 03:01, 18 June 2019 (UTC)

It wasn't "fine", it was an exploit. Just not one that was exploited before, and not one that allowed it to run on unapproved/unflagged accounts. Headbomb {t · c · p · b}

Speaking with my BAG hat on, I absolutely oppose a full bot run being made from the logged-in user’s account. Even more so if there is no means to kill the run. Since I was brought here by a ping to IABot, I will say my tool does not execute bot jobs from the user’s own account. It will only make edits on their account with the single page tool. Bot jobs can also be killed. This is what I would like to see if user submitted jobs are going to be implemented.—CYBERPOWER (Chat) 15:48, 18 June 2019 (UTC)
- I will see about writing the code to change it. I wrote half the code last night. AManWithNoPlan (talk) 15:59, 18 June 2019 (UTC)
  - It's not only being able to kill jobs in the case of a malfunction, it's also that these edits and runs get performed from unflagged accounts, being able to destructively run on large amounts of pages that the bot was never meant to be run on, and can cause major WP:BOTDICT#flooding issues (both in watchlists/recent changes), as well as prevent WP:HIDEBOTS from being usable. Headbomb {t · c · p · b} 16:09, 18 June 2019 (UTC)

So, having the user id and bot switching places in the edit summary is the end of the world. I could add code to make the bot check if itself is blocked. AManWithNoPlan (talk) 16:53, 18 June 2019 (UTC)

It is, yes. In the sense that individual users are not authorized to run fully-automated bot processes on their own accounts for a plethora of reasons, one of them being flooding. Headbomb {t · c · p · b} 17:01, 18 June 2019 (UTC)

That is 100% wrong understanding of how this works. The bot will not be “running” on accounts. It simply saves the edits using someone else’s UID. There is no ability for anyone to run the bot faster. It all runs on one server. AManWithNoPlan (talk) 17:10, 18 June 2019 (UTC)

This is a fully automated bot running on my account. That the processing happens on an external server somewhere is irrelevant. Headbomb {t · c · p · b} 17:22, 18 June 2019 (UTC)

This is a "fully-automated bot" running on Sandstein's account. This is a "fully-automated bot" running on Plastikspork's account. Yet no one has accused either of those admins of violating bot policy in the time since those edits were made, and no one has accused the developer of the script they used of violating bot policy. This means that what you call a "fully-automated bot" is not something that is as prohibited as you are making it out to be. * Pppery * _{it has begun...} 17:55, 18 June 2019 (UTC)

Those are scripts, not bots. And yes, there is a difference. Headbomb {t · c · p · b} 18:44, 18 June 2019 (UTC)

What is the difference between XFDcloser and Citation bot that makes one a script and the other a bot. The only relevant difference is that Citation bot's processing happens on an external server somewhere as opposed to in JavaScript, which you said above is irrelevant. * Pppery * _{it has begun...} 20:26, 18 June 2019 (UTC)

Pppery, XfDCloser is a JS script invoked by the user to run on a select amount of pages and can be killed by simply reloading Tempe browser, is easily reverted, and is semi-automated. The user is in full control and any damage is easily undone by reverting their own edit and trying again. Citation Bot would run on an inappropriately high number of pages, is completely automated and cannot be killed. Also anything that floods a watchlist or recent changes needs a BRFA. —CYBERPOWER (Around) 11:56, 19 June 2019 (UTC)

Pppery, there is a difference. When a user engages the scripts on their account, they are taking responsibility for every edit it makes. It's their own script after all and they can kill it whenever. Initiating a job with someone else's potentially malfunctioning bot programming, with no way to kill it leaves that user's account running rampant with no control from the user. They will be blocked, but cannot be expected to take responsibility for something that is not in their control. This in effect makes it the bot operator's responsibility. This is precisely why bot jobs are done from IABot's account and only individual edits are made from the user's account. And in the event of a malfunction, the issue is confined to a single account. —CYBERPOWER (Chat) 19:12, 18 June 2019 (UTC)

Does revoking the OAuth consumer behind Citation bot's access to your account not count? * Pppery * _{it has begun...} 20:26, 18 June 2019 (UTC)

Pppery, It might, but I'm not sure about active sessions. —CYBERPOWER (Chat) 23:53, 18 June 2019 (UTC)

You can filter your watch list to no show the citation bot based upon the tags. AManWithNoPlan (talk) 17:37, 18 June 2019 (UTC)

"it's also that these edits and runs get performed from unflagged accounts, being able to destructively run on large amounts of pages that the bot was never meant to be run on" -- how is that any different whether the bot is set to write as itself or as a user? Are there pages that are in some way flagged to not allow bots to edit? AManWithNoPlan (talk) 17:37, 18 June 2019 (UTC)

Because the operator bears responsibility and is trusted to have these powers. Random Editor 124824 is not. Or socks. Or vandals with an account. Headbomb {t · c · p · b} 17:55, 18 June 2019 (UTC)

The random user has no more ability to make or control edits with the bot when it edits as them than when it is edits as the bot. They have no increased powers. AManWithNoPlan (talk) 18:00, 18 June 2019 (UTC)

That is a besides the point, they can still unleash the bot in an automated mode across thousands of pages without the bot flag, without oversight, without community trust, against policy, and without BRFAs. If you want to pin this on Citation bot, Citation bot is not approved to run in this manner either. Headbomb {t · c · p · b} 18:38, 18 June 2019 (UTC)

AManWithNoPlan, there are serious exploits you are leaving with your implementation, one that cannot be approved. Headbomb lays it out pretty well. Your current implementation is too open to many forms of abuse including, I can probably DoS the shit out of your server by spawning so many bot jobs that the server runs out resources. I can create a huge batch of sockpuppets and launch a job with each one, especially when there's a widespread bug in the bot, and disruptively break a lot of articles thus giving admins a hard time cleaning up after the mess by blocking every sock account and then reverting every edit. Not to mention I can flood other users' watchlists because I will be running the bots from non-bot accounts. An unsuspecting user trying to do good may run the bot, and actually cause huge problems and now can't stop it. This forces us to block the account until the job is finished and that poor user is now responsible for something they shouldn't have to be. And yes there are {{nobots}} templates your bot should be adhering to. —CYBERPOWER (Chat) 19:20, 18 June 2019 (UTC)

I am curious how any of those things would be changed by having the edits by flagged as the bot instead of the user? "DoS the shit"-- no different. "batch of sockpuppets" -- no different. " flood other users' watchlists" -- no different, since you can remove our edits from your watch list using tags. "This forces us to block the account" -- actually it is very easy to block the bot as a whole, please see the blocking page. "And yes there are {{nobots}}" -- no different. AManWithNoPlan (talk) 19:45, 18 June 2019 (UTC)

Then propose a BRFA for that and gain consensus for this mode of a operation. This will require an RFC because no BAG member would approve this as is. Headbomb {t · c · p · b} 20:02, 18 June 2019 (UTC)

This isn't a bot; it doesn't need a BRFA. * Pppery * _{it has begun...} 20:26, 18 June 2019 (UTC)

It is a bot. See WP:BOTDICT#Bot vs WP:BOTDICT#Script if you need clarification on what these words mean. Headbomb {t · c · p · b} 20:44, 18 June 2019 (UTC)

Pppery, It is a bot. Anything automated without supervision is a bot. Scripts are supervised by the account owners. This isn't. I will reject this proposal at any BRFA unless I see a discussion where the community finds this to be acceptable. —CYBERPOWER (Chat) 23:56, 18 June 2019 (UTC)

AManWithNoPlan, why should users be forced to apply a tag to the watchlist filter? If every bot op did that, we'd have a lot of pissed off users on Wikipedia because they have to filter out every individual bot. You are no exception. Use a bot flagged account and leave the users filtering bot edits as a whole in peace. —CYBERPOWER (Chat) 23:58, 18 June 2019 (UTC)

I have talked to Smith and he liked it the old way, so I should have code deployed soon. I have been pushing the discussion hard to try to figure out what is really troubling people, and all I have gotten is "it is better when the bot does it", which is enough if Smith likes it that way. AManWithNoPlan (talk) 20:04, 18 June 2019 (UTC)

I, on the other hand, think that it is better for edits made directly on behalf of a user not to be proxied through bot accounts. * Pppery * _{it has begun...} 20:26, 18 June 2019 (UTC)

It's rather bizarre to see requests that the edits are made with the Citation bot user ID when said account is still blocked, ostensibly to avoid block evasion. The continuing block seems a very clear message from the administrators that they want the edits to be made under individual users' accounts, presumably for additional accountability. I don't see why the bot operators should change anything when the admins are giving mixed signals. Nemo 06:06, 19 June 2019 (UTC)
Nemo bis, No, that’s it at all. The bot has security problems and it simply needs to disallow edits from LTAs. Make the bot more secure and add the name of the summoner of the bot in an edit summary, CitationBot will be unblocked.—CYBERPOWER (Around) 12:00, 19 June 2019 (UTC)

Someone needs to unblock the bot, because the security problem is fixed by forcing edits to be done as the user. We are working on conversion to writing as the bot but with verifiable user names because of people who don’t like seeing bots on their watchlist. Either way, the need for the block is gone. AManWithNoPlan (talk) 12:42, 19 June 2019 (UTC)

there is no ETA on the bot ever working again.AManWithNoPlan (talk) 23:52, 19 June 2019 (UTC)

I don't code, and have never been involved in the development of a bot. I've been watching this controversy at a distance and I'm not sure the reason for the confusion. AFAICT, there have been recent concerns that the bot can be invoked by editors but there is no accounting of who is invoking it. The suggested solution for this has always been to use OAUTH to authenticate the editor invoking the bot, and then to record this info probably in an edit summary. It was never suggested that the bot should be run from editor's accounts. The reasons for this are explained above, but even if people involved in the development of the bot don't understand or don't accept these reasons, until and unless they can convince the community otherwise that isn't going anywhere.

That some of those involved in the development of the bot didn't understand the proposal was to use OAUTH to record the editor running it rather than to run it from the editors account is unfortunate, especially the time they wasted doing the latter. But it doesn't change it was always the intention and clear to at least some of us with no involvement and most importantly it doesn't change the fact it's not an acceptable practice.

Now after the concerns about a lack of authentication were raised, someone probably an editor blocked for harassment began to use the bot to follow 2 editors who'd they'd previously been harassing in the same way. We can't know for sure who did this, but that is part of the problem and these sort of concerns are why the lack of authentication was considered a problem. Adding authentication would partly solve the problem. While running the bot from user accounts may also solve the problem, it's still not an acceptable solution for the earlier reasons.

I said "partly" because as I understand it, there's nothing to stop a blocked editor from invoking the account. (I think only a globally locked account since they can't login.) Probably the best solution for this would be for the bot to check if the editor invoking it is currently blocked in the wikipedia where it's being invoked and to refuse to function if this is the case. While it's possible an internal blacklist of editors or something similar would work, this seems to be adding unnecessary complexity. It may also be wise to consider getting the bot to check confirmed status and similar in case it needs to be limited to such editors in the future.

Nil Einne (talk) 05:28, 20 June 2019 (UTC)

the blocked user check was implemented before oauth was rolled out, but it never mattered since the Bot was blocked and Wikipedia employees stepped up to the plate and implemented oauth—although in a way that ended up pissing people off. Filtering out blocked accounts is easy. AManWithNoPlan (talk) 17:11, 21 June 2019 (UTC)

AManWithNoPlan, Wikipedia employees? —CYBERPOWER (Chat) 00:59, 22 June 2019 (UTC)

https://github.com/davidbarratt AManWithNoPlan (talk) 01:16, 22 June 2019 (UTC)

Am I missing something? How can you reliably filter out blocked accounts if you do not have a mechanism of verifying who is actually activating the bot?

Also I don't understand how "it never mattered". From what I saw at AN, the bot was blocked precisely because it was being used by someone, possibly a blocked user, to follow 2 editors who had previously been followed by this blocked user. Stopping this from being repeated was something that mattered to a lot of people, hopefully including everyone working on the bot. The suggestion at AN as here was to ensure WP:BOTMULTIOP was followed and that the identity of who activated the bot was both disclosed and verified. It was never suggested at AN that it was acceptable to violate bot policy by operating bots from accounts without approval. As I said, somewhat separately it would be better to check the person activating it was not blocked since otherwise you'd need to do something more complex to stop it being misused even with verification and disclosure.

As said at the beginning, I have no idea how on earth you could do this reliably if you didn't actually verify who was activating it. So while you could I guess implement the code concurrently, AFAICT you would need to complete the verification step before it was any use. It doesn't sound to me like it should have been that hard to implement, but since I don't code and know nothing about the APIs, I don't know. Also unless I'm confused such changes weren't necessary anyway if the bot couldn't be directed by other users but instead could only be run on their accounts. I'm sure blocks prevent editing via the API as well. But that doesn't change the problems arising from running bots on unsanctioned accounts.

Ultimately from what I saw at AN, if these changes had actually been implemented correctly there's a fair chance the block would now be unblocked. Assuming no one, whether the WMF or whatever, is willing to work on complying with BOTMULTIOP I'm not sure where we go. Since there was some tolerance on allowing the bot to run in the manner it did before, maybe people could be convinced to do a trial unblock sometimes in the future to see if whoever is misusing it is still paying attention. If people are too concerned or it is unblocked and misuse continues, I guess the bot will need to stay blocked until someone is willing to work on a solution which comply without our bot policies and guidelines.

I'm still somewhat confused about the WMF's involvement. My read GitHub is work began long before the WMF got involved. Are they actually the ones who decided to make the bot run from unsanctioned accounts or did they just follow what was already in the works thinking that those involved knew what they were doing? In any case, it seems largely a moot point for this discussion, the fact remains the change wasn't acceptable for reasons outlined above who made the changes doesn't affect that.

If the WMF are actually the ones who decided to implement the changes in that way, this seems to suggest a communication failure. While I can understand why they probably didn't want to read that whole AN thread and may not entirely understand our bot policy, they probably should have checked better with us what was needed before making the changes. But that's a discussion best held elsewhere. (If they just followed what was already going on, or one of us told them it was what we wanted, then it's at least partly on us.)

Nil Einne (talk) 11:13, 22 June 2019 (UTC)

There doesn't need to be these massive walls of text and blame/investigation/whatever here. Someone understood something differently than intended, in good faith. They made the bot do something, in good faith, that didn't complain with policy/consensus. The bot no longer does this, and what is desired has been clarified. Now it's just a matter of waiting for bot maintainers / WMF people / random passerbys to figure out how to do that. Headbomb {t · c · p · b} 16:02, 22 June 2019 (UTC)

Arbitrary break

Reading the above discussion in more depth, I wonder if the 'check for block' point has been misunderstood. I was never suggesting the bot needs to check if it itself is block. I guess this is useful if the bot is allowed to operate from random unsanctioned accounts, but that is still a problem for other reasons so was never likely to be an acceptable solution. My point was the bot needed to check that the user activating was not blocked since otherwise the misuse would be harder to stop even with disclosure and verification of the user activating. If the bot is only running from its own account as it should be, then checking whether it is blocked is not quite so important. Probably the bot does need some sort of basic sanity check otherwise it could break when it is unblocked if it doesn't understand it was blocked before and its edits failed. (It could just completely break when it is blocked and needed to be manually killed or something. And flooding the servers with denied requests may be a problem from the WMF's POV. But those don't really affect us directly here.) In any case, if there's still a lot of misunderstanding over what's being request going on, maybe it would be helpful if people summarise somewhere what they plan to do. In sufficient detail so there should be no confusion but not so much that no one will read it. Then hopefully someone with more knowledge will have the time to say whether it's right or not. Nil Einne (talk) 11:46, 22 June 2019 (UTC)

Not sure if you this helps you exactly, but some of the points you talked about (i.e. blocks) above are discussed here: https://github.com/ms609/citation-bot/issues/1767 by the operator/maintainers/WMF staff. You can also follow the work/progress there, might be a lot of technical stuf in it but seems to provide a lot of answers to points above. Redalert2fan (talk) 11:58, 22 June 2019 (UTC)

checking to see if the user is blocked is really easy. That was implemented within hours of the block going into effect. That was enough to unblock the bot, but since we had almost finished oauth, we held off. Oauth was implemented where the user was the person given credit in the edit summary and the description and tag was where the bot got credit. People flipped out and now we are swapping those (the Bot gets credit for the edit and the user gets put in the edit summary). This will in no way effect who can initiate the bot and in no way effect what pages the bot can edit and in no way effects what edits were made. All it will change is the edit summary. This edit summary change does effect filtering of edits and watch lists and such, so it’s not just pickiness. AManWithNoPlan (talk) 17:52, 22 June 2019 (UTC)

Fails to properly update to cite journal

Status: {{fixed}}
Reported by: Headbomb {t · c · p · b} 19:57, 19 June 2019 (UTC)

What happens: [28]
What should happen: [29]
We can't proceed until: Feedback from maintainers

Further hdl resolvers

Status: {{fixed}}
Reported by: Headbomb {t · c · p · b} 03:40, 20 June 2019 (UTC)

What should happen: [30], also [31], also [32], also [33], also [34]
We can't proceed until: Feedback from maintainers

https://github.com/ms609/citation-bot/pull/1827 AManWithNoPlan (talk) 23:08, 24 June 2019 (UTC)

further postscript cleanup

As well as

|postscript= (as detailed here)

also cover

|postscript=<!-- Bot inserted parameter. Either remove it; or change its value to "." for the cite to end in a ".", as necessary. -->

Headbomb {t · c · p · b} 00:30, 21 June 2019 (UTC)

Support ol=

Status: {{fixed}}
Reported by: Headbomb {t · c · p · b} 04:56, 21 June 2019 (UTC)

What should happen: [35]
We can't proceed until: Feedback from maintainers

https://github.com/ms609/citation-bot/pull/1829 AManWithNoPlan (talk) 23:21, 24 June 2019 (UTC)

Fails to expand when there's markup in the title

If you have

{{cite journal |title=Selenium Chloride and Bromide Equilibria in Aprotic Solvents; A <sup>77</sup>Se NMR Study |journal=Polyhedron |year=1990 |volume=9 |pages=589–595 |doi=10.1016/S0277-5387(00)86238-5}}
"Selenium Chloride and Bromide Equilibria in Aprotic Solvents; A ⁷⁷Se NMR Study". Polyhedron. 9: 589–595. 1990. doi:10.1016/S0277-5387(00)86238-5.

the bot won't expand it.

However, if you have

{{cite journal |journal=Polyhedron |year=1990 |volume=9 |pages=589–595 |doi=10.1016/S0277-5387(00)86238-5}}
Polyhedron. 9: 589–595. 1990. doi:10.1016/S0277-5387(00)86238-5. {{cite journal}}: Missing or empty |title= (help)

then the bot will expand it to

Lamoureux, Marc; Milne, John (1990). "Selenium chloride and bromide equilibria in aprotic solvents; a 77Se NMR study". Polyhedron. 9 (4): 589–595. doi:10.1016/S0277-5387(00)86238-5.

Title mismatches aren't important when the DOI is provided. Headbomb {t · c · p · b} 17:11, 21 June 2019 (UTC)

https://github.com/ms609/citation-bot/pull/1828 should help a lot AManWithNoPlan (talk) 23:13, 24 June 2019 (UTC)

{{fixed}} AManWithNoPlan (talk) 00:44, 25 June 2019 (UTC)

Better handling of crap inputs

Status: {{fixed}}
Reported by: Headbomb {t · c · p · b} 21:51, 22 June 2019 (UTC)

What happens: [36]
What should happen: Same + [37]
We can't proceed until: Feedback from maintainers

Title 404 - Not Found

Status: {{fixed}}
Reported by: Redalert2fan (talk) 18:15, 24 June 2019 (UTC)

What happens: Title including "404 - Not Found" was added
Relevant diffs/links: [38]
We can't proceed until: Feedback from maintainers

Invalid date -0001-11-30

Status: {{fixed}}
Reported by: Redalert2fan (talk) 19:01, 24 June 2019 (UTC)

What happens: date -0001-11-30 is added
Relevant diffs/links: [39]
We can't proceed until: Feedback from maintainers

Note that apart from the year being 0001 which is quite unlikely the date starts with a - and dates should obviously not be negative. Redalert2fan (talk) 19:01, 24 June 2019 (UTC)

Actually that’s just a “before Christ” date.🙄🤪. I thought we filtered those out. AManWithNoPlan (talk) 19:22, 24 June 2019 (UTC)

Ah of course, I forgot about B.C. being possible. I wonder if they had citeweb at the time. hmmm... --Redalert2fan (talk) 19:25, 24 June 2019 (UTC)

interestingly enough, for ancient manuscripts you can get citations with B.C. Dates. AManWithNoPlan (talk) 21:08, 24 June 2019 (UTC)

Citation bot should not render any dates like that unless Citation bot has an agreement with whatever/whoever is the source of the date. This because that format appears to be ISO 8601 which requires such agreements for dates that are not Gregorian dates (roughly before 1582). And, even if Citation bot does have such an agreement, cs1|2 does not so will reject such dates.

—Trappist the monk (talk) 23:11, 24 June 2019 (UTC)

Dates like 1540-06-12 don't require any agreements with any source or organization to mean 12 June 1540. The only ambiguity is in what '12 June 1540' date this refers to, the 12 June 1540 in the calendar in use at the time, or the 12 June 1540 in the Gregorian calendar.

We can choose to not use that format because some people will assume that it implies an ISO 8601 date. But it's a local choice we made, not a choice imposed on us by external requirements.

The real issue here we made that choice, and the format isn't supported. Headbomb {t · c · p · b} 02:53, 25 June 2019 (UTC)

AdsAbs again

AdsAbs is giving the 999 error again, It seems we have used up all our uses. Is it correct that we have 25000 in total per x amount of time? I think I saw a counter on the web page while running a list of pages. My apologies to everyone else, I probably used 15000 of those haha. Redalert2fan (talk) 10:15, 7 June 2019 (UTC)

25000 searches. AManWithNoPlan (talk) 13:03, 7 June 2019 (UTC)

Seems it has reset for today, lets see how fast its used up this time. 1/25000 was used between 19:52 and 19:55 according to web page log on my current run. Redalert2fan (talk) 20:19, 7 June 2019 (UTC)

1 hour later and past 7000 searches. Redalert2fan (talk) 20:59, 7 June 2019 (UTC)

After 4 hours we are just above 22000 searches, so we have nearly run out again. Rate has slightly gone down since the reset but that seems to be due to only me using the bot. While usage may be higher than before due to the unblock, and me currently running the list that I have basically saved up probably does not help, it is not out of the question that multiple users on a "normal" day could also use up all of the searches. --Redalert2fan (talk) 00:05, 8 June 2019 (UTC)

it was really bad back when the test suite queried it. AManWithNoPlan (talk) 00:59, 8 June 2019 (UTC)

Limit busted again. When I ran bibcode bot, they gave me unlimited rates, so I don't see why they couldn't give unlimited rates to this bot either. Headbomb {t · c · p · b} 02:56, 25 June 2019 (UTC)

that almost sounds like a “back in my day...” story. 🤣😂 How many times do you query per day? We are capped at 25,000 queries. AManWithNoPlan (talk) 03:19, 25 June 2019 (UTC)

I discovered that we were doing the search on complete citations and we were doing it on some citations twice :-( I have fixed that. AManWithNoPlan (talk) 15:32, 25 June 2019 (UTC)

flagging as {{fixed}} since that is gonna help a lot. AManWithNoPlan (talk) 15:32, 25 June 2019 (UTC)

Minimizing queries is always good! And the bot was never super stable, so it could rarely do long uninterrupted runs, but I routinely processed several thousands/tens of thousands of articles per day, so 25000 queries is not something that would have been unusual. The bot also ran a few times per month, rather than continuously.

The folks at NASA were always glad to help Wikipedia though, so it may just be a matter of contacting them and ask for a higher limit (e.g. 100,000 queries). Headbomb {t · c · p · b} 15:48, 25 June 2019 (UTC)

I think the fact that we run 24/7 made them say no. They are requiring us the make changes before they will double it. AManWithNoPlan (talk) 16:39, 25 June 2019 (UTC)

Could be. Wonder what changes they have in mind. Headbomb {t · c · p · b} 17:06, 25 June 2019 (UTC)

Please update bibcodes that change

Status: {{fixed}} and will reduce queries long-term
Reported by: Headbomb {t · c · p · b} 19:56, 19 June 2019 (UTC)

What should happen: [40]
We can't proceed until: Feedback from maintainers

When a certain bibcode (e.g. Bibcode:2018MNRAS.tmp.2192I) is valid but redirects to a new bibcode (e.g. Bibcode:2018MNRAS.481..703I) the bibcode should be updated. Headbomb {t · c · p · b} 19:56, 19 June 2019 (UTC)

obviously only ones with tmp in them to avoid getting banned. AManWithNoPlan (talk) 00:46, 22 June 2019 (UTC)

Actually no, there's a lot more than just those the tmp ones, and it's always safe to updatee to the most recent / active value. User:Bibcode Bot used to do that, but it's currently down because of API changes at ADSABS. Headbomb {t · c · p · b} 03:14, 22 June 2019 (UTC)

will investigate api to see if we can do this without using any extra api calls AManWithNoPlan (talk) 04:41, 22 June 2019 (UTC)

we do detect this in the big search API, and bail out and have to individually search all the bibcodes one by one, since this messes up the search results. I will have to look at the bibcode output for these bibcodes and see what we can do. AManWithNoPlan (talk) 17:29, 25 June 2019 (UTC)

If this requires additional API queries, and we're query limited, then you could limit this to bibcodes with 'tmp' in them, as well as those with 'arXiv' in them. Headbomb {t · c · p · b} 20:24, 25 June 2019 (UTC)

Non-human authors from books.google.com

Status: {{fixed}}
Reported by: Redalert2fan (talk) 15:37, 21 June 2019 (UTC)

What happens: last2 = Of Anthropology first2 = Stanford University. Dept
What should happen: should not be added.
Relevant diffs/links: [41]
We can't proceed until: Feedback from maintainers

FDA website

Status: GIGO {{fixed}}, will continue discussion in bug below
Reported by: QuackGuru (talk) 03:45, 25 June 2019 (UTC)

What happens: FDA citations errors
What should happen: Not sure the best way to format FDA citations
Relevant diffs/links: https://en.wikipedia.org/w/index.php?title=User%3AQuackGuru%2FSand3&type=revision&diff=903275857&oldid=903239164
We can't proceed until: Feedback from maintainers

This will help once deployed https://github.com/ms609/citation-bot/pull/1833/files#diff-47b867068e1aef0388448a571c8c7092 AManWithNoPlan (talk) 05:03, 25 June 2019 (UTC)

that helped a lot. And now this should do it https://github.com/ms609/citation-bot/pull/1834 AManWithNoPlan (talk) 15:29, 25 June 2019 (UTC)

The bot is running very slow. I am trying to activate it. QuackGuru (talk) 17:47, 25 June 2019 (UTC)

I cannot even submit patches as this time. AManWithNoPlan (talk) 19:40, 25 June 2019 (UTC)

What can be done to supercharge the bot. Maybe a clone bot. Two bots can do the same job. QuackGuru (talk) 19:46, 25 June 2019 (UTC)

The page is massive, the bot will always be slow on such pages. Headbomb {t · c · p · b} 20:23, 25 June 2019 (UTC)

Google books

Need to expand based upon URL, Chapter-URL, and ChapterURL. AManWithNoPlan (talk) 17:51, 25 June 2019 (UTC)

https://github.com/ms609/citation-bot/pull/1838 {{fixed}} AManWithNoPlan (talk) 22:09, 25 June 2019 (UTC)

remove via when identifiers are there

Status: {{fixed}}
Reported by: Headbomb {t · c · p · b} 21:24, 25 June 2019 (UTC)

What should happen: [42]
We can't proceed until: Feedback from maintainers

https://github.com/ms609/citation-bot/pull/1841 AManWithNoPlan (talk) 22:19, 25 June 2019 (UTC)

Looking at that, it's not quite doing what's requested. The request is for any |via= parameter when there is any identifier (save for ISSN, which is crap, and points nowhere useful) and no url. Basically, in line with the relevant part of the documentation "when the URL provided does not make clear the identity of the deliverer". Headbomb {t · c · p · b} 22:28, 25 June 2019 (UTC)

That's gonna get us flamed really bad. I know, I know, but people love their via. I got told off for removing via=JSTOR when the only link was a jstor parameter. AManWithNoPlan (talk) 22:36, 25 June 2019 (UTC)

don't capitalize inside html tags

Status: {{fixed}}
Reported by: Headbomb {t · c · p · b} 21:29, 25 June 2019 (UTC)

Relevant diffs/links: [43]
We can't proceed until: Feedback from maintainers

https://github.com/ms609/citation-bot/pull/1840 AManWithNoPlan (talk) 22:10, 25 June 2019 (UTC)

APIs not working: ISBN

Status: new bug
Reported by: Eastmain (talk • contribs) 00:27, 9 June 2019 (UTC)

What happens: also, when running Citation bot on article Intersex

  ! Did not receive results from Google API search isbn:9783941957503

That ISBN corresponds to https://frl.publisso.de/resource/frl:5226608

We can't proceed until: Feedback from maintainers

"Did not receive results from Google API search isbn:9783941957503" There is nothing we can do about books Google does not have. The Hive Mind is not all knowing yet. AManWithNoPlan (talk) 04:05, 9 June 2019 (UTC)

{{notabug}} AManWithNoPlan (talk) 13:54, 26 June 2019 (UTC)

better cite thesis support

Status: {{fixed}}
Reported by: Headbomb {t · c · p · b} 20:59, 25 June 2019 (UTC)

What should happen: [44]
We can't proceed until: Feedback from maintainers

Note the comma removal, and the url handling. Headbomb {t · c · p · b} 20:59, 25 June 2019 (UTC)

Just URL handling. Might look at comma's and such later. https://github.com/ms609/citation-bot/pull/1849 AManWithNoPlan (talk) 23:27, 26 June 2019 (UTC)

AdsAbs/bibcode

The API is completely busted. AManWithNoPlan (talk) 23:42, 25 June 2019 (UTC)

https://github.com/ms609/citation-bot/pull/1845 AManWithNoPlan (talk) 19:45, 26 June 2019 (UTC)

More betterly volume/issue cleanup

Status: {{fixed}}
Reported by: Headbomb {t · c · p · b} 01:47, 26 June 2019 (UTC)

What should happen: [45]
We can't proceed until: Feedback from maintainers

Please add more proxy removal

Status: {{fixed}}
Reported by: Nemo 06:01, 6 June 2019 (UTC)

What happens: Proxy survives.
What should happen: special:diff/900519852
We can't proceed until: Feedback from maintainers

We don't remove them unless we can fix them or there is a doi present. AManWithNoPlan (talk) 13:51, 6 June 2019 (UTC)

So the solution to the specific case is to make it recognise such ezproxy subdomain as equivalent to ieeexplore.ieee.org? For some reason they used hyphens instead of dots. Nemo 06:24, 7 June 2019 (UTC)

proxies are a personal vendetta of mine. I will work on adding more. AManWithNoPlan (talk) 13:07, 7 June 2019 (UTC)

I will look around and figure out what proxies are the most command and regex them first. AManWithNoPlan (talk) 16:25, 16 June 2019 (UTC)

considering how many times I have manually removed them; I am surprised I didn’t do more of this sooner. AManWithNoPlan (talk) 00:50, 18 June 2019 (UTC)

https://github.com/ms609/citation-bot/pull/1852 AManWithNoPlan (talk) 14:52, 27 June 2019 (UTC)

More URL redundancy

Status: new bug
Reported by: Headbomb {t · c · p · b} 23:04, 22 June 2019 (UTC)

What should happen: [46] and [47]
We can't proceed until: Feedback from maintainers

🤔 tool server right now is having some serious black listing problems. Might be related. I know some of the reported issues actually work just find on the test cluster. AManWithNoPlan (talk) 02:37, 23 June 2019 (UTC)

No rush. Headbomb {t · c · p · b} 02:46, 23 June 2019 (UTC)

Is it a rate limiting or something confined to the current server's IP address, in your opinion? Maybe restarting on another host machine on Toolforge will help? Nemo 13:41, 23 June 2019 (UTC)

{{fixed}} as best we can. One of them detect AE are not hunan . AManWithNoPlan (talk) 12:35, 27 June 2019 (UTC)

Diacritics in Szegő

Status: feature request {{wontfix}}
Reported by: Nemo 13:00, 26 June 2019 (UTC)

What should happen: special:diff/903568213
We can't proceed until: Feedback from maintainers

I see that crossref returns Szeg{ö and that makes me wonder if there is any way to know what the { means in a name with an reliablity. AManWithNoPlan (talk) 19:44, 26 June 2019 (UTC)

I see that the websites of the journals have ö and not the ő displayed. The jstor linked one has ordinary o. I strongley suspect that this is unfixable GIGO. AManWithNoPlan (talk) 23:20, 26 June 2019 (UTC)

Headbomb Thoughts? AManWithNoPlan (talk) 23:28, 26 June 2019 (UTC)

GIGO is GIGO. If you want to code an exception to handle this specific case, go for it. Headbomb {t · c · p · b} 00:14, 27 June 2019 (UTC)

Adding superfluous DOI when JSTOR is present

Status: {{notabug}}, since the DOIs actually sometimes get their target changed to something else, and most tools only understand DOIs, and lastly WP:OVERLINK explicitly excludes citations.
Reported by: bender235 (talk) 16:48, 27 June 2019 (UTC)

What happens: The bot adds DOI like doi:10.2307/2224098 when the equivalent JSTOR 2224098 is present. This creates unnecessary clutter. It seems this has been pointed out eight years ago, too.
Relevant diffs/links: Special:Diff/882639329
We can't proceed until: Feedback from maintainers

This is not a bug. Both identifiers are distinct, even if they point to the same page. The DOI could well be updated to point to a different page if a different publishing agreement is ever stuck, and various external tools expect a DOI over a JSTOR identifier. Both belong. Headbomb {t · c · p · b} 16:57, 27 June 2019 (UTC)

Indeed there are thousands of 10.2307 DOIs which nowadays point to external publishers, and conversely there are thousands JSTOR IDs with no corresponding valid 10.2307 DOI. So it's a useful information to have. Nemo 17:37, 27 June 2019 (UTC)

If the DOI does not work, then we remove it. The fact that they often don’t point to the same page is nuts. AManWithNoPlan (talk) 17:52, 27 June 2019 (UTC)

What do you mean by "external tools expect a DOI"? Does this concern displayed DOI, or just the (hidden) Z39.88 elements? --bender235 (talk) 19:46, 27 June 2019 (UTC)

I mean that researchers use a plethora of computer programs and applets, and some of them are built around DOIs. Headbomb {t · c · p · b} 19:48, 27 June 2019 (UTC)

Replacing ISBN-10 with ISBN-13 for old books

Status: {{fixed}} (the style manual changed, and we have changed with it).
Reported by: bender235 (talk) 16:56, 27 June 2019 (UTC)

What happens: The bot replaces ISBN 0-444-10097-0 with ISBN 978-0-444-10097-9 for books published long before ISBN-13 became standard. Since both expressions are technically equivalent, this is not necessarily a bug if someone could point me at the discussion where consensus for this has been established. So far I've been very meticulous to always add original ISBN-10 for books that were published with those.
Relevant diffs/links: Special:Diff/882639329
We can't proceed until: Feedback from maintainers

This was the Wikipedia standard and then it wasn’t. The bot now only does it for 2007 and later. I should note that using dashes is now the standard too, but we don’t mess with that. AManWithNoPlan (talk) 17:42, 27 June 2019 (UTC)

What does "now" mean? Because in the linked diff, it changed ISBNs for books published in the 1990s and 1970s. --bender235 (talk) 19:40, 27 June 2019 (UTC)

Now, as in after February 2019, which is when that diff is from. Headbomb {t · c · p · b} 19:45, 27 June 2019 (UTC)

Ok, thanks. To be clear, I don't mind the changes to ISBN-13 as long as we're consistent. I'd just like to know what the current style guide is on this issue. --bender235 (talk) 19:47, 27 June 2019 (UTC)

https://en.wikipedia.org/wiki/Wikipedia:ISBN This style guide. So, you are supposed to use ISBN13, if it exists. Everything 2007 and forward as ISBN13. The bot checks the date and then converts it, if the date is 2007 or newer. This is a change to the old guidance which was to convert them all. AManWithNoPlan (talk) 20:50, 27 June 2019 (UTC)

More via cleanup

Status: {{fixed}} what we can without getting flamed. Why people flame this, I do not know.
Reported by: Headbomb {t · c · p · b} 21:00, 27 June 2019 (UTC)

What should happen: [48]
We can't proceed until: Feedback from maintainers

Interestingly, that citation has a PMID, an ISBN, and a URL. They all point to different end urls about the same thing. So, the via actually is applicable enough for the bot to leave. You as a human have judgement and can remove it. AManWithNoPlan (talk) 01:16, 28 June 2019 (UTC)

They all point to the same thing, the method of access is irrelevant. And it causes issues with things like this because a pointless |via= is kept when new URLs are added and |via= isn't updated, or pointlessly kept around. Headbomb {t · c · p · b} 04:07, 28 June 2019 (UTC)

Better italics handling

Status: {{fixed}}
Reported by: Headbomb {t · c · p · b} 00:15, 31 May 2019 (UTC)

What should happen: [49]
We can't proceed until: Feedback from maintainers

[[ANTIC (magazine)|''ANTIC'']] AManWithNoPlan (talk) 02:29, 31 May 2019 (UTC)

Because of style variations we do not remove italics from titles. AManWithNoPlan (talk) 14:43, 31 May 2019 (UTC)

Run it on

Harari, H. (1975). "A new quark model for hadrons". Physics Letters B. 57 (3): 265–269. Bibcode:1975PhLB...57..265H. doi:10.1016/0370-2693(75)90072-6. {{cite journal}}: Italic or bold markup not allowed in: |journal= (help)

You'll see the bot does that already, and has done so for years. CS1/2 has italicized works, always. Headbomb {t · c · p · b} 15:03, 31 May 2019 (UTC)

oops, I was thinking title, not works. AManWithNoPlan (talk) 17:59, 31 May 2019 (UTC)

Cleanup authorn= + firstn=

When you have something like

{{cite journal |author1=Vennes |first1=S. |last2=Kawka |first2=A. |year=2008 |title=On the empirical evidence for the existence of ultramassive white dwarfs |journal=Monthly Notices of the Royal Astronomical Society |volume=389 |issue=3 |pages=1367 |arxiv=0806.4742 |bibcode=2008MNRAS.389.1367V |doi=10.1111/j.1365-2966.2008.13652.x }}

It should be cleaned up to

{{cite journal |last1=Vennes |first1=S. |last2=Kawka |first2=A. |year=2008 |title=On the empirical evidence for the existence of ultramassive white dwarfs |journal=Monthly Notices of the Royal Astronomical Society |volume=389 |issue=3 |pages=1367 |arxiv=0806.4742 |bibcode=2008MNRAS.389.1367V |doi=10.1111/j.1365-2966.2008.13652.x }}

Headbomb {t · c · p · b} 16:55, 16 June 2019 (UTC)

Done. https://github.com/ms609/citation-bot/pull/1861 {{fixed}}. AManWithNoPlan (talk) 17:05, 28 June 2019 (UTC)

PDF in title cleanup?

Status: {{fixed}}
Reported by: Headbomb {t · c · p · b} 23:18, 18 June 2019 (UTC)

What should happen: [50]
We can't proceed until: Feedback from maintainers

Not sure how safe this is, but it's worth investigation. I believe we're doing similar things with PDF at the start of the title though. Headbomb {t · c · p · b} 23:18, 18 June 2019 (UTC)

https://github.com/ms609/citation-bot/pull/1863 I cannot think of any reason to not do this. AManWithNoPlan (talk) 18:25, 28 June 2019 (UTC)

US FDA website

Status: {{fixed}}
Reported by: QuackGuru (talk) 18:10, 25 June 2019 (UTC)

What should happen: The FDA website is not a journal. The citations already state "United States Food and Drug Administration" under "agency". Stating simply FDA is ambiguous. There are many countries that have an FDA.
Relevant diffs/links: https://en.wikipedia.org/w/index.php?title=User%3AQuackGuru%2FSand3&type=revision&diff=903437656&oldid=903337104
We can't proceed until: Feedback from maintainers

That use of agency is wrong though. agency: The news agency (wire service) that provided the content; examples: Associated Press, Reuters, Agence France-Presse. May be wikilinked if relevant. --Izno (talk) 18:25, 25 June 2019 (UTC)

The bot did not fix that. QuackGuru (talk) 18:27, 25 June 2019 (UTC)

Original: "FDA's New Regulations for E-Cigarettes, Cigars, and All Other Tobacco Products". United States Department of Health and Human Services. United States Food and Drug Administration. 17 August 2017.

Agency removed: "FDA's New Regulations for E-Cigarettes, Cigars, and All Other Tobacco Products". United States Food and Drug Administration. 17 August 2017.

What is the best way to reformat this citation? I would like the bot to fix it without me trying to fix it. QuackGuru (talk) 18:28, 25 June 2019 (UTC)

I do know that |agency= refers to wire services like Reuters and not government agencies. AManWithNoPlan (talk) 19:32, 25 June 2019 (UTC)

Can the bot remove agency and move "United States Food and Drug Administration" under publisher while also removing "United States Department of Health and Human Services" under publisher? This effects several articles I added United States Food and Drug Administration under agency. It is too many citations to manually to fix. QuackGuru (talk) 19:38, 25 June 2019 (UTC)

It would be a special case, but we can do that. AManWithNoPlan (talk) 19:39, 25 June 2019 (UTC)

I'll try again when the bot is updated. I would also like the bot to not change it to "cite journal" or add "journal=". QuackGuru (talk) 19:44, 25 June 2019 (UTC)

It changed the FDA citations to "cite journal".[51] QuackGuru (talk) 04:05, 28 June 2019 (UTC)

This will fix that for FDA.GOV and quite a few other websites. https://github.com/ms609/citation-bot/pull/1864 once this says "merged". I have to wait for it to pass tests. AManWithNoPlan (talk) 18:37, 28 June 2019 (UTC)

should be fixed now. Cleaning existing references not yet done. AManWithNoPlan (talk) 19:29, 28 June 2019 (UTC)

Okay. I just activated the bot. QuackGuru (talk) 19:50, 28 June 2019 (UTC)

At the bottom of the page it says "No changes required." I wanted the bot to remove agency and move "United States Food and Drug Administration" under publisher while also removing "United States Department of Health and Human Services". There are numerous citations that are in mainspace that could be updated. QuackGuru (talk) 20:15, 28 June 2019 (UTC)

As I said. That part is not done yet. AManWithNoPlan (talk) 20:17, 28 June 2019 (UTC)

Sorry. I misunderstood. QuackGuru (talk) 20:20, 28 June 2019 (UTC)

Sorry, QuackGuru, I was not clear. The final fix is now uploaded and tested. AManWithNoPlan (talk) 21:02, 28 June 2019 (UTC)

It worked! QuackGuru (talk) 21:29, 28 June 2019 (UTC)

Citation bot shortens first names

Citation bot should not shorten first names, as it was not always easy to find what that first name was. Urhixidur (talk) 12:53, 28 June 2019 (UTC)

I assume it is this: edit you are referring to? --Redalert2fan (talk) 13:16, 28 June 2019 (UTC)

Can you please point me to an edit where the bot converted Floor to F., since I cannot find one. The bot could very well add "F.", if that is all the database has. AManWithNoPlan (talk) 14:26, 28 June 2019 (UTC)

{{notabug}} The guy's name is "F." on the publication. If we were going to make a guess as to his first name, I can think of plenty of other F-words 😂 https://www.aanda.org/articles/aa/abs/2007/41/aa8357-07/aa8357-07.html AManWithNoPlan (talk) 18:18, 28 June 2019 (UTC)

More garbage volume/issues/pages

Status: {{fixed}}
Reported by: Headbomb {t · c · p · b} 07:45, 27 June 2019 (UTC)

What should happen: [52], [53], [54], [55], [56], [57]
We can't proceed until: Feedback from maintainers

This is not garbage. It's a more transparent form of citation and conflicts with WP:CITEVAR. The bot should never do this.--Fuhghettaboutit (talk) 01:55, 28 June 2019 (UTC)

the templates exist for multiple reasons: COINS data, easy bot manipulation, standard formatting, etc. The use of vol. and such violates all these principles and the template documentation. AManWithNoPlan (talk) 03:37, 28 June 2019 (UTC)

WP:CITEVAR says: “fixing errors in citation coding, including incorrectly used template parameters, and <ref> markup problems: an improvement because it helps the citations to be parsed correctly;”, which supports the bot’s actions AManWithNoPlan (talk) 03:41, 28 June 2019 (UTC)

Surgeon General citations

Status: new bug
Reported by: QuackGuru (talk) 21:51, 28 June 2019 (UTC)

What should happen: I would like the bot to remove agency and move "United States Food and Drug Administration" under publisher while also removing "United States Department of Health and Human Services" for the E-cigarettes.surgeongeneral.gov website.
We can't proceed until: Feedback from maintainers

"THE FACTS on e-cigarette use among youth and young adults". United States Department of Health and Human Services. Surgeon General of the United States. 2016.

I did a page search for "United States Department of Health and Human Services". I just noticed the same problem with the Surgeon General citations. I restored the issue for one of the citations for the bot to undo. QuackGuru (talk) 21:51, 28 June 2019 (UTC)

did the bot miss anything. AManWithNoPlan (talk) 00:21, 29 June 2019 (UTC)

For the FDA website and the E-cigarettes.surgeongeneral.gov website everything is fixed.

The last mistake you found is the cdph.ca.gov website. I would like the bot to do this for several articles. I don't remember all the articles I made mistakes. QuackGuru (talk) 02:39, 29 June 2019 (UTC)

I would like the bot to undo this edit. QuackGuru (talk) 02:43, 29 June 2019 (UTC)

try it now. AManWithNoPlan (talk) 18:41, 29 June 2019 (UTC)

I tried running it twice. The bot is stalled. QuackGuru (talk) 00:04, 30 June 2019 (UTC)

repeatedly running it on the same page results in multiple workers doing the same thing, and just makes it worse. The bot is currently under heavy usage. Marking as {{fixed}}, since it worked in my tests. AManWithNoPlan (talk) 01:22, 30 June 2019 (UTC)

Move volume/issue/pages thing

Status: {{fixed}}
Reported by: Headbomb {t · c · p · b} 18:02, 30 June 2019 (UTC)

What should happen: [58]
We can't proceed until: Feedback from maintainers

Caps: I/i

Status: {{fixed}} mostly by adding a bunch of Russian words
Reported by: Headbomb {t · c · p · b} 18:13, 30 June 2019 (UTC)

What happens: [59] [Ruch Prawniczy, Ekonomiczny i Socjologiczny → Ruch Prawniczy, Ekonomiczny I Socjologiczny ]
What should happen: Leave I alone. There's just too many russian/polish/etc journals with 'i' in it.
We can't proceed until: Feedback from maintainers

Missed a vol/issue

Status: {{fixed}}
Reported by: Headbomb {t · c · p · b} 02:27, 1 July 2019 (UTC)

What happens: [60]
What should happen: same+[61]
We can't proceed until: Feedback from maintainers

We don’t always do it, since some things should have volume/vol in the text (volume is part of the actual title for many books and should not use volume parameter, but they do). Will adjust code for this case. AManWithNoPlan (talk) 04:02, 1 July 2019 (UTC)

volume is part of the actual title for many books and should not use volume parameter, but they do Where is this assertion coming from? I routinely remove volume from title. --Izno (talk) 12:03, 1 July 2019 (UTC)

Well, if you have something like The Art of Problem Solving, Volume 2: And Beyond ISBN 978-0-9773045-8-5, that would be such a case. Headbomb {t · c · p · b} 13:35, 1 July 2019 (UTC)

More volume cleanup

Status: {{fixed}}
Reported by: Headbomb {t · c · p · b} 06:13, 2 July 2019 (UTC)

What should happen: [62]+[63]+[64]+[65]+[66] (cite mag)
We can't proceed until: Feedback from maintainers

More redundant URLs

Status: {{wontfix}} but don’t know why curl doesn’t work
Reported by: Headbomb {t · c · p · b} 07:28, 2 July 2019 (UTC)

What should happen: [67] + [68]
We can't proceed until: Feedback from maintainers

ResearchGate URL cleanup

Status: {{wontfix}}
Reported by: Headbomb {t · c · p · b} 22:12, 2 July 2019 (UTC)

What should happen: [69], or possibly shove those in a {{cite web}}/{{cite journal}} and try to expand Headbomb {t · c · p · b} 22:12, 2 July 2019 (UTC)
We can't proceed until: Feedback from maintainers

It already tries to this. If no expansion is successful, then we revert to the original. Sometime ResearchGate lets us in, and often not. AManWithNoPlan (talk) 22:19, 2 July 2019 (UTC)

The shortening would still be good though. Headbomb {t · c · p · b} 23:16, 2 July 2019 (UTC)

I don't think so, if it is just a plain link. If it is a plain link, you can tell what the link is "going to", and if they ever die, you can search for an alterative archvie based on the metadata in the URL. Only if other text to help identify the subject of the URL is present should the link be shortened. (t) Josve05a (c) 23:18, 2 July 2019 (UTC)

Hmm... That's a valid point. Would be nice to at least have have the trimming in

<ref>[https://www.researchgate.net/publication/23907078_Far-infrared_sources_in_the_vicinity_of_the_supernova_remnant_W28 Bla bla bla]</ref>

→ <ref>[https://www.researchgate.net/publication/23907078 Bla bla bla]</ref> though. Not critical however. Headbomb {t · c · p · b} 06:05, 3 July 2019 (UTC)

Perhaps we could change them from https://www.researchgate.net/publication/23907078_Far-infrared_sources_in_the_vicinity_of_the_supernova_remnant_W28 to https://www.researchgate.net/publication/23907078_ResearchGate_needs_to_unblock_citation_bot_NOW AManWithNoPlan (talk) 16:37, 3 July 2019 (UTC)

not gonna fix, since I do not trust or want to use the URL phrases, since it is not reliable. AManWithNoPlan (talk) 18:37, 3 July 2019 (UTC)

Weird... failure of cite web --> cite journal?

Status: {{fixed}}
Reported by: Headbomb {t · c · p · b} 06:01, 3 July 2019 (UTC)

What happens: [70]
What should happen: [71]
We can't proceed until: Feedback from maintainers

That’s really weird. AManWithNoPlan (talk) 13:16, 3 July 2019 (UTC)

It's this old code : if (strpos($this->get('doi'), '10.1093') !== FALSE) return; That predates the creation of the check_10_1093_doi() function. 10.1093 DOIs require special love care and feeding. AManWithNoPlan (talk) 15:21, 3 July 2019 (UTC)