Ensure original URLs are redirected (301 Moved Permanently) to normalized names, including pages that has redirects

[renoirb]

Once we have #3 addressed, we’ll have to configure web server to make proper HTTP redirection for funny URL names and others that may create conflicts (e.g. inconsistent casing #2)

Handle cases such as:

• tutorials/What is CSS? into tutorials/What_is_CSS
• concepts/accessibility/definitions/visual impairment into concepts/accessibility/definitions/visual_impairment
• html/elements/!DOCTYPE into html/elements/DOCTYPE
• svg/properties/animVal (SVGAnimatedString) into svg/properties/animVal_SVGAnimatedString
• css/Properties/color into css/properties/color (notice "Properties" casing here)
• css/properties/flex-align into css/properties/align-items (notice casing here too)

Objectives

1. Keep original URLs in place
2. Redirect to a valid URL that removes potential confusion (ref: URL "code-points", and RFC3986 at 3.3. Path)
3. Make obvious mapping from URL into a file name in the source repository

Expected deliverables

• List, and configure for NGINX, all deleted pages to their configured redirects (redirects.txt)
• List, and configure in a format suitable for NGINX, all URLs we want to "sanitize" (sanity_redirects.txt)
• Commit NGINX configuration options into https://github.com/webplatform/docs along with directory listing configuration from Find a way to reproduce sub page listing without relying on static site generator #3

Reference

• NGINX redirect map

[renoirb]

URLs to test out.

Those are ones with quirky URLS MediaWiki was allowing that we must redirect.

To consider

• use of %20 for space (and NOT +) has to be supported
• use of _ as an interchangeable token for a space
  because its most likely that links in content might have been written like that and we can catch that case easily.

Questions

• Should we support + as a space too?

Should handle URL with invalid character components to a valid location

• http://67.205.56.184/wiki/tutorials/What%20can%20you%20do%20with%20JavaScript
  • http://67.205.56.184/wiki/tutorials/What_can_you_do_with_JavaScript
  • http://67.205.56.184/wiki/tutorials/What+can+you+do+with+JavaScript won’t work; is that OK?
• http://67.205.56.184/wiki/svg/elements%20(alphabetical%20order)
  • http://67.205.56.184/wiki/svg/elements+(alphabetical+order) won’t work; is that OK?
  • http://67.205.56.184/wiki/svg/elements_(alphabetical_order)
• http://67.205.56.184/wiki/css/selectors/pseudo-classes/:target
• http://67.205.56.184/wiki/css/functions/translate3d()
• http://67.205.56.184/wiki/css/atrules/@viewport
• http://67.205.56.184/wiki/html/attributes/href%20(base)
  • http://67.205.56.184/wiki/html/attributes/href_(base)
• http://67.205.56.184/wiki/html/attributes/type%20type%20(a,%20link,%20embed)
  • http://67.205.56.184/wiki/html/attributes/type_type_(a,_link,_embed)
• http://67.205.56.184/wiki/html/elements/!DOCTYPE
• http://67.205.56.184/wiki/css/selectors/pseudo-elements/::after
• http://67.205.56.184/wiki/css/selectors/pseudo-classes/:lang(c)
• http://67.205.56.184/wiki/css/selectors/pseudo-classes/:nth-child(n)
• http://67.205.56.184/wiki/html/attributes/value%20(param%20element)
  • http://67.205.56.184/wiki/html/attributes/value_(param_element)

Should handle redirects of deleted content with redirects left behind

Based on a few entries from reports/summary.yml that has a "redirect_to" flag.

• http://67.205.56.184/wiki/canvas
• http://67.205.56.184/wiki/concepts/Design%20theory
• http://67.205.56.184/wiki/tutorials/Programming_-_the_real_basics (should go to "concepts/programming/programming_basics/")
• http://67.205.56.184/wiki/css/Properties/color (should go to "css/properties/color")
• http://67.205.56.184/wiki/css/Properties/opacity (should go to "css/properties/opacity")
• canvas tutorial section
  • http://67.205.56.184/wiki/tutorials/canvas/Canvas_tutorial/Compositing

In the following links; Does it really has links "with a space" in the page content (e.g. "[[canvas/tutorial/Canvas tutorial/Drawing shapes]]"), or only with an underscore (e.g. "[[canvas/tutorial/Canvas_tutorial/Drawing_shapes]]")

• http://67.205.56.184/concepts/creating_prototypes
  • http://67.205.56.184/wiki/tutorials/creating%20prototypes ?
• http://67.205.56.184/tutorials/canvas/Canvas_tutorial
  • http://67.205.56.184/wiki/canvas/tutorial/Canvas%20tutorial/Basic%20usage
• http://67.205.56.184/tutorials/canvas/Canvas_tutorial/Drawing_shapes
  • http://67.205.56.184/wiki/canvas/tutorial/Canvas_tutorial/Drawing_shapes
  • http://67.205.56.184/wiki/canvas/tutorial/Canvas%20tutorial/Drawing%20shapes
• http://67.205.56.184/concepts/Internet_and_Web/how_browsers_work
  • http://67.205.56.184/wiki/tutorials/how_browsers_work
  • http://67.205.56.184/wiki/concepts/Internet_and_Web/how_browsers%20work
  • http://67.205.56.184/wiki/concepts/Internet_and%20WeB/how_BroWsers%20work

Should give message to disabled features

• show a "Feature disabled" page.
• return the same response body as requested (Accept: text/html to return HTML document, Accept: application/json to return a JSON string and so on)

Tests:

• http://67.205.56.184/wiki/User:Shepazu
• http://67.205.56.184/wiki/Special:FooBarBazz
• http://67.205.56.184/w/api.php?action=query&meta=siteinfo&siprop=statistics

Redirect WPD pages into migrated content

• Link in template should have new location already
  • http://67.205.56.184/wiki/WPD:Community
  • http://67.205.56.184/wiki/WPD:Contributors_Guide

Should handle translation and open the appropriate content

• http://67.205.56.184/Main_Page/zh-hant
• http://67.205.56.184/wiki/concepts/Internet%20and%20Web/How%20does%20the%20Internet%20Work/es
• http://67.205.56.184/wiki/concepts/Internet_and_Web/The_Web_Standards_Model/ja
  • http://67.205.56.184/wiki/concepts/Internet%20and%20Web/The%20Web%20Standards%20Model/ja
• http://67.205.56.184/wiki/css/ko
• http://67.205.56.184/Main_Page/zh-hant

Should redirect anything /wiki/foo to /foo only at the end

• http://67.205.56.184/wiki/tutorials/games%20onslaught
• http://67.205.56.184/wiki/tutorials/design
• http://67.205.56.184/wiki/tutorials/canvas%20entanglement
  • http://67.205.56.184/wiki/tutorials/canvas_entanglement
  • http://67.205.56.184/wiki/tutoRials/canvAs_entanglEmenT

Should sanitize URLs

• "tutorials/What is CSS?"

    http head 'http://67.205.56.184/wiki/tutorials/What_is_CSS?'
    HTTP/1.1 301 Moved Permanently
    // ...
    Location: http://67.205.56.184/tutorials/What_is_CSS
  

  • http://67.205.56.184/wiki/tutorials/What%20is%20Css?
  • http://67.205.56.184/wiki/tutorials/What%20IS%20css?
  • http://67.205.56.184/wiki/tutorials/what_is_css? notice the ? and the inconsistent caSinG!
  • http://67.205.56.184/wiki/tutorials/What%20is%20CSS? (notice the ?, and yet its not a GET query parameter marker!)
  • http://67.205.56.184/wiki/tutorials/What_is_CSS?

Should be case InSeNsiTive

... and redirect at the same place location

http://67.205.56.184/wiki/apis/webrtc/objects/MediaStream/properties/videoTracks

• http://67.205.56.184/wiki/apis/webrtc/objects/mediastream/properties/videotracks
• http://67.205.56.184/wiki/APIS/WebRTC/Objects/mediaStream/Properties/videotrackS
  • http://67.205.56.184/wiki/APIs/WeBRTC/ObjeCts/mEDiaStream/PropertiEs/viDeoTrackS

http://67.205.56.184/wiki/Main_Page :

• http://67.205.56.184/wiki/main_page
• http://67.205.56.184/wiki/main%20pAge

[renoirb]

The redirect map is generated through the app/console mediawiki:summary and creates reports/nginx_redirects.map that we’ll have loaded in NGINX configuration.

[renoirb]

Some more URLs to try

Main content

• http://67.205.56.184/wiki/tutorials/what_is_css
  • http://67.205.56.184/wiki/tutorials/what%20is%20css
• http://67.205.56.184/wiki/WPD:Contributors_Guide
• http://67.205.56.184/wiki/WPD:Community
• http://67.205.56.184/wiki/apis/xhr/methods/open%20(XDomainRequest)
• http://67.205.56.184/wiki/css/selectors/pseudo-classes/:target
• http://67.205.56.184/wiki/concepts/Internet_and_Web/The_Web_Standards_Model/ja
• http://67.205.56.184/wiki/css/atrules/@viewPort
  • http://67.205.56.184/css/atrules/viewPort
• http://67.205.56.184/wiki/css/functions/translate3d()
  • http://67.205.56.184/css/functions/translate3d
• http://67.205.56.184/wiki/css/properties/backface-visibility ... and links from page:
  • http://67.205.56.184/wiki/css/concepts/initial_value
  • http://67.205.56.184/wiki/css/concepts/inherited
  • http://67.205.56.184/wiki/css/concepts/computed_value
  • http://67.205.56.184/wiki/css/concepts/cssom
  • http://67.205.56.184/wiki/css/cssom/MSCSSMatrix/methods/multiply
• http://67.205.56.184/wiki/html/attributes/href%20(base)

WPD namespace

• http://67.205.56.184/wiki/WPD:Tasks
• http://67.205.56.184/WPD:Tasks
• http://67.205.56.184/wiki/WPD:Sandbox
  • http://67.205.56.184/WPD:Sandbox
• http://67.205.56.184/wiki/WPD:Infrastructure
  • http://67.205.56.184/WPD:Infrastructure
• http://parsoid.webplatform.org/wiki/WPD:Getting_Started/examples

[renoirb]

Let’s make HTML some MediaWiki ASK query results.

Migrated tasks originally noted here into webplatform/docs#2

Converted:

• http://parsoid.webplatformstaging.org/wiki/WPD:Getting_Started/examples/api
• http://parsoid.webplatformstaging.org/wiki/WPD:Getting_Started/examples/css

To review

• http://parsoid.webplatformstaging.org/wiki/WPD:Projects/api_docs
• http://parsoid.webplatformstaging.org/wiki/WPD:Getting_Started

[renoirb]

Most use cases should work now!