Bug 329528 - Support Markdown in WikiText
Summary: Support Markdown in WikiText
Status: RESOLVED FIXED
Alias: None
Product: z_Archived
Classification: Eclipse Foundation
Component: Mylyn (show other bugs)
Version: unspecified   Edit
Hardware: All All
: P3 enhancement with 6 votes (vote)
Target Milestone: 1.8.0   Edit
Assignee: Stefan Seelmann CLA
QA Contact:
URL: http://daringfireball.net/projects/ma...
Whiteboard:
Keywords: contributed, noteworthy, plan
Depends on:
Blocks: 404095 404096 409682
  Show dependency tree
 
Reported: 2010-11-05 07:17 EDT by Alex Blewitt CLA
Modified: 2015-04-02 06:01 EDT (History)
19 users (show)

See Also:


Attachments
Adds basic markdown support: inline HTML, escaping, paragraphs, line breaks, headers, blockquotes, code blocks, emphasis (231.12 KB, patch)
2013-02-10 09:28 EST, Stefan Seelmann CLA
no flags Details | Diff
mylyn/context/zip (3.25 KB, application/octet-stream)
2013-03-25 12:27 EDT, David Green CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Alex Blewitt CLA 2010-11-05 07:17:28 EDT
WikiText looks to be a great standard for representing wiki text formats. It's especially useful if you have one or more already in existence.

However, I'm surprised that Markdown isn't mentioned (or supported). StackOverflow uses markdown for its wiki syntax and is increasinly popular as both a text-readble and convertable format. There's also real-time previews for such syntaxes, like wmd-editor.com as well as StackOverflow's fork of the same.

Lastly, it also supports inline HTML where necessary, code blocks, and conversion to pretty-printed data. In fact, this entire comment is valid Markdown, even if I use *emphasis* or other HTML.

One key benefit of Markdown over the existing textile support is with headers. Compare and contrast:

h1. A header

# A Header #

A Header
======

All of these will work the same way. You can also have nested headers too.

Finally, code samples (with indented spaces) work better with MArkdown than others. For example:

    // Java code example
    for(int i=0;i<10;i++)
      System.println("I doesn't need escaping");
Comment 1 David Green CLA 2010-11-08 15:00:27 EST
Alex, thanks for the feature request.  We would welcome any contribution that introduces support for Markdown.
Comment 2 Alex Blewitt CLA 2011-01-31 06:41:48 EST
Can you give pointers on how/where to get started with such a request?
Comment 3 David Green CLA 2011-01-31 15:40:46 EST
A great place to start is the WikiText Developer Guide, specifically the section on "Markup Languages":http://help.eclipse.org/helios/index.jsp?topic=/org.eclipse.mylyn.wikitext.help.ui/help/devguide/Markup-Languages.html
If you have any questions after you've looked there, be sure to ask.
Comment 4 Frank Gerhardt CLA 2012-01-04 09:41:15 EST
GitHub uses Markdown in their open-source wiki server, https://github.com/github/gollum 

They have some extensions though http://github.github.com/github-flavored-markdown/

On Android there is a nice app called WikiMind that also uses Markdown, https://market.android.com/details?id=com.sweetrain.wikimind

It would be great to have Markdown support in WikiText and use the same files all over the place, on GitHub, in Eclipse and on Android.
Comment 5 Torkild Resheim CLA 2012-02-29 05:20:09 EST
On OS X there is a great app for Markdown editing called Mou. http://mouapp.com/.
Comment 6 Benjamin Cabé CLA 2012-07-06 11:26:38 EDT
We just opened CQ 6632 for reusing Java MarkDown [1] in Lua Development Tools.
This might be of interest to implement MD support in WikiText!

[1] https://github.com/cletus/jmd
Comment 7 David Green CLA 2012-07-09 14:20:03 EDT
Thanks for the pointer Benjamin, that's great.  It should be pretty easy to wrap another MD implementation for use in Mylyn WikiText to generate output.  The rich editor in Eclipse relies on the markup parser to provide character offsets to the document partitioner, token scanner etc.  It would be interesting to see if it was feasible to get this level of integration out of Java MarkDown.
Comment 8 Stefan Seelmann CLA 2012-10-15 17:33:14 EDT
I started with an implementation here: https://github.com/seelmann/mylyn.docs/tree/markdown. Currently only default paragraph and atx-style headers are working. It may take some time till it is usable as I only work on it in my spare time. I'll provide a patch then. Contributions are welcome :)


The first problem I faced are setext-style headers (http://daringfireball.net/projects/markdown/syntax#header) where headers are "underlined" using equal signs (for first-level headers) and dashes (for second-level headers). For example:

This is an H1
=============

This is an H2
-------------

When processing the "This is an H1" I need to lookahead the next line. Does the WikiText framework currently support that?
Comment 9 Nepomuk Seiler CLA 2012-10-19 07:05:33 EDT
Is this maybe done with the "PatternBasedElementProcessor". I just scrolled a bit in the source code?

What is your current status? I will try to implement the emphasize ( _emphasize this_) and bold ( **this is bold** ) blocks.
Comment 10 Stefan Seelmann CLA 2012-10-19 14:20:52 EDT
Great, thanks for helping to implement the Markdown support :)

I found a solution for the underlined headers. I created a LookAheadReader that is used to read the next line. 

The emphasis and bold phrases are already implemented. Here is the list of features that are already implemented:
* Inline HTML 
* Paragraphs
* Headers
* Blockquotes
* Code Blocks
* Horizontal LInes
* Emphasis
* Inline Code	
* Backslash Escapes

I'm currentliy working on 
* Lists

TODOs:
* Links
* Images
* Automatic Links
* Extensions (Table etc.)
Comment 11 Stefan Seelmann CLA 2013-02-10 09:28:06 EST
Created attachment 226807 [details]
Adds basic markdown support: inline HTML, escaping, paragraphs, line breaks, headers, blockquotes, code blocks, emphasis

I didn't manage to implement more features, but want to share the current work. I think it makes sense to include the current state into wikitext.

From the TOC at http://daringfireball.net/projects/markdown/syntax, the following features are implemented or still to do:

* Overview
** Inline HTML: done
** Automatic Escaping for Special Characters: done
* Block Elements
** Paragraphs and Line Breaks: done
** Headers: done
** Blockquotes: done
** Lists: TODO
** Code Blocks: done
** Horizontal Rules: done
* Span Elements
** Links: TODO
** Emphasis: done
** Code: done
** Images: TODO
* Miscellaneous
** Backslash Escapes: done
** Automatic Links: TODO
Comment 12 David Green CLA 2013-02-13 20:13:19 EST
Stefan, thanks for the contribution!  I've pushed your attachment to Gerrit to make it easier to provide feedback:

10364: 329528: Support Markdown in WikiText [If7acf06d]
https://git.eclipse.org/r/#/c/10364/
Comment 13 David Green CLA 2013-03-13 18:41:47 EDT
I mistakenly created the review against the incubator.  With the Creole language contribution we added the language implementation to the org.eclipse.mylyn.docs Git repo, and added it to the WikiText extras feature.  I'd like to do this with Markdown as well, since we don't have a good way of consuming features from the incubator.

Stefan, if you agree could you abandon the original review and post your latest changeset as a review against the docs repository instead?  ssh://dgreen@git.eclipse.org:29418/mylyn/org.eclipse.mylyn.docs
Comment 14 Stefan Seelmann CLA 2013-03-13 19:06:19 EDT
Sure David, I'll submit the patchset against docs repo. However I don't see the "Abandon" button in Gerrit, seems I don't have permissions, can you do that?
Comment 15 Stefan Seelmann CLA 2013-03-13 20:57:59 EDT
I pushed it to https://git.eclipse.org/r/#/c/11144/

Here is my contributor statement:
1. I have authored 100% of the content I'm contributing (except .project and .settings which I copied from another wikitext project)
2. I have the rights to donate the content to Eclipse
3. I contribute the content under the EPL
Comment 16 David Green CLA 2013-03-20 18:17:58 EDT
filed CQ 7134
Comment 17 David Green CLA 2013-03-22 15:46:42 EDT
CQ 7134 has been approved.

discussion regarding inclusion in the Mylyn WikiText Extras feature: http://dev.eclipse.org/mhonarc/lists/mylyn-docs-dev/msg00162.html

pushed review: https://git.eclipse.org/r/11398
Comment 18 David Green CLA 2013-03-25 12:27:46 EDT
the following reviews have been merged:
* https://git.eclipse.org/r/11144
* https://git.eclipse.org/r/11398
* https://git.eclipse.org/r/11432
Available now from the Mylyn Docs nightly build for those that would like to try it out.  p2 site here: https://hudson.eclipse.org/hudson/job/mylyn-docs-nightly/lastSuccessfulBuild/artifact/org.eclipse.mylyn.docs-site/target/site/

Thanks for getting this one off the ground Stefan.
Comment 19 David Green CLA 2013-03-25 12:27:53 EDT
Created attachment 229003 [details]
mylyn/context/zip
Comment 20 Steffen Pingel CLA 2013-05-02 07:09:06 EDT
Please use this update site when installing nightly builds of Mylyn Docs: http://download.eclipse.org/mylyn/snapshots/nightly/docs/.