codehilite extension double escapes HTML

[kkinder]

Here's an example:

import markdown

example_content = """# Test 
  
    >>> print('hi') 
    hi 

The above is valid MarkDown."""

output = markdown.markdown(example_content, extensions=['markdown.extensions.codehilite'])
print(output)

As you can see, my example file includes a Markdown example with > characters. Unfortunately, with Codehilite enabled, it produces doubly-escaped content. Here's the output of that script:

<h1>Test</h1>
<div class="codehilite"><pre><span></span>&amp;gt;&amp;gt;&amp;gt; print(&#39;hi&#39;) 
hi
</pre></div>


<p>The above is valid MarkDown.</p>

See those &> parts? That means your HTML output includes the > characters escaped twice. It should be just >>>, not >>>.

[facelessuser]

I'll take a look at this today when I get some time.

[kkinder]

Thanks. I created a pull request that fixes it a quick and dirty way. Since the extensions seem to get markup after it's been escaped, this seemed like the only obvious way to resolve the problem short of refactoring how Python-Markdown processes extensions.

[facelessuser]

I'll have to take a look, because this may unescape intentional escaped, literal syntax.

[facelessuser]

I don't think this used to be the case. I imagine this was introduced in recent refactoring or pull request. I'll get to the bottom of this though.

[facelessuser]

Yes, I introduced this I think when adjusting escapes in the serializer. We now escape the code when it is intially found by the code block processors. There should be no risk in unescaping literals. The proper approach is probably unescaping the content before processing it. I'll confirm in a bit.