Quantcast

[PATCH 1 of 4] minirst: detect bullet lists using asterisks

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[PATCH 1 of 4] minirst: detect bullet lists using asterisks

Gregory Szorc
# HG changeset patch
# User Gregory Szorc <[hidden email]>
# Date 1487205737 28800
#      Wed Feb 15 16:42:17 2017 -0800
# Node ID dd90d5f7dc1908d9b69e6a4b8165a73757d1c84b
# Parent  afaf3c2b129c8940387fd9928ae4fdc28259d13c
minirst: detect bullet lists using asterisks

Previously, the "bullet" regular expression excluded the asterisk
('*') as a character denoting a bulleted list. Why I'm not sure
because the asterisk seems to be the canonical bullet character
in reST these days.

This patch makes asterisk-prefixed lines parse as bulleted lists.

diff --git a/mercurial/minirst.py b/mercurial/minirst.py
--- a/mercurial/minirst.py
+++ b/mercurial/minirst.py
@@ -138,7 +138,7 @@ def findliteralblocks(blocks):
         i += 1
     return blocks
 
-_bulletre = re.compile(r'(-|[0-9A-Za-z]+\.|\(?[0-9A-Za-z]+\)|\|) ')
+_bulletre = re.compile(r'(\*|-|[0-9A-Za-z]+\.|\(?[0-9A-Za-z]+\)|\|) ')
 _optionre = re.compile(r'^(-([a-zA-Z0-9]), )?(--[a-z0-9-]+)'
                        r'((.*)  +)(.*)$')
 _fieldre = re.compile(r':(?![: ])([^:]*)(?<! ):[ ]+(.*)')
diff --git a/tests/test-minirst.py b/tests/test-minirst.py
--- a/tests/test-minirst.py
+++ b/tests/test-minirst.py
@@ -118,6 +118,13 @@ Line blocks are also a form of list:
 | This is the first line.
   The line continues here.
 | This is the second line.
+
+Bullet lists are also detected:
+
+* This is the first bullet
+* This is the second bullet
+  It has 2 lines
+* This is the third bullet
 """
 
 debugformats('lists', lists)
diff --git a/tests/test-minirst.py.out b/tests/test-minirst.py.out
--- a/tests/test-minirst.py.out
+++ b/tests/test-minirst.py.out
@@ -187,6 +187,12 @@ Line blocks are also a form of list:
 
 This is the first line. The line continues here.
 This is the second line.
+
+Bullet lists are also detected:
+
+* This is the first bullet
+* This is the second bullet It has 2 lines
+* This is the third bullet
 ----------------------------------------------------------------------
 
 30 column format:
@@ -231,6 +237,14 @@ list:
 This is the first line. The
 line continues here.
 This is the second line.
+
+Bullet lists are also
+detected:
+
+* This is the first bullet
+* This is the second bullet It
+  has 2 lines
+* This is the third bullet
 ----------------------------------------------------------------------
 
 html format:
@@ -276,6 +290,14 @@ Line blocks are also a form of list:
  <li> This is the first line.   The line continues here.
  <li> This is the second line.
 </ol>
+<p>
+Bullet lists are also detected:
+</p>
+<ol>
+ <li> This is the first bullet
+ <li> This is the second bullet   It has 2 lines
+ <li> This is the third bullet
+</ol>
 ----------------------------------------------------------------------
 
 == options ==
_______________________________________________
Mercurial-devel mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[PATCH 2 of 4] minirst: dynamically compile admonitions regexp

Gregory Szorc
# HG changeset patch
# User Gregory Szorc <[hidden email]>
# Date 1487188034 28800
#      Wed Feb 15 11:47:14 2017 -0800
# Node ID 197ba3e5885366038d453b9b22fa2910a0792988
# Parent  dd90d5f7dc1908d9b69e6a4b8165a73757d1c84b
minirst: dynamically compile admonitions regexp

Currently, parsing admonitions uses a static regular expression created
from a pre-defined list of admonitions. A future patch will introduce a
feature that needs to parse custom admonitions. Prepare for this by
compiling the admonitions regular expression during each function
invocation.

Strictly speaking, there is a slight performance loss here. But we only
run this code as part of displaying help text. I don't think the loss
will be noticeable and I don't think we care if it were.

diff --git a/mercurial/minirst.py b/mercurial/minirst.py
--- a/mercurial/minirst.py
+++ b/mercurial/minirst.py
@@ -411,18 +411,31 @@ def prunecomments(blocks):
             i += 1
     return blocks
 
-_admonitionre = re.compile(r"\.\. (admonition|attention|caution|danger|"
-                           r"error|hint|important|note|tip|warning)::",
-                           flags=re.IGNORECASE)
+
+_admonitions = set([
+    'admonition',
+    'attention',
+    'caution',
+    'danger',
+    'error',
+    'hint',
+    'important',
+    'note',
+    'tip',
+    'warning',
+])
 
 def findadmonitions(blocks):
     """
     Makes the type of the block an admonition block if
     the first line is an admonition directive
     """
+    admonitionre = re.compile(r'\.\. (%s)::' % '|'.join(sorted(_admonitions)),
+                              flags=re.IGNORECASE)
+
     i = 0
     while i < len(blocks):
-        m = _admonitionre.match(blocks[i]['lines'][0])
+        m = admonitionre.match(blocks[i]['lines'][0])
         if m:
             blocks[i]['type'] = 'admonition'
             admonitiontitle = blocks[i]['lines'][0][3:m.end() - 2].lower()
_______________________________________________
Mercurial-devel mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[PATCH 3 of 4] minirst: support passing admonitions into findadmonitions() and parse()

Gregory Szorc
In reply to this post by Gregory Szorc
# HG changeset patch
# User Gregory Szorc <[hidden email]>
# Date 1487188152 28800
#      Wed Feb 15 11:49:12 2017 -0800
# Node ID 9ab3ad4934aedc649d28f74dd70207c6c6e88596
# Parent  197ba3e5885366038d453b9b22fa2910a0792988
minirst: support passing admonitions into findadmonitions() and parse()

This will allow consumers to declare a custom list of admonitions
to parse. Without this patch, custom admonitions would get removed
when prunecomments() is run. We could add an argument controlling
whether prunecomments() is run. However, it is better to convert
the "paragraph" block to an "admonition" block so consumers don't
have to parse for custom admonitions.

diff --git a/mercurial/minirst.py b/mercurial/minirst.py
--- a/mercurial/minirst.py
+++ b/mercurial/minirst.py
@@ -425,12 +425,14 @@ def prunecomments(blocks):
     'warning',
 ])
 
-def findadmonitions(blocks):
+def findadmonitions(blocks, admonitions=None):
     """
     Makes the type of the block an admonition block if
     the first line is an admonition directive
     """
-    admonitionre = re.compile(r'\.\. (%s)::' % '|'.join(sorted(_admonitions)),
+    admonitions = admonitions or _admonitions
+
+    admonitionre = re.compile(r'\.\. (%s)::' % '|'.join(sorted(admonitions)),
                               flags=re.IGNORECASE)
 
     i = 0
@@ -642,7 +644,7 @@ def formathtml(blocks):
 
     return ''.join(out)
 
-def parse(text, indent=0, keep=None):
+def parse(text, indent=0, keep=None, admonitions=None):
     """Parse text into a list of blocks"""
     pruned = []
     blocks = findblocks(text)
@@ -657,7 +659,7 @@ def parse(text, indent=0, keep=None):
     blocks = splitparagraphs(blocks)
     blocks = updatefieldlists(blocks)
     blocks = updateoptionlists(blocks)
-    blocks = findadmonitions(blocks)
+    blocks = findadmonitions(blocks, admonitions=admonitions)
     blocks = addmargins(blocks)
     blocks = prunecomments(blocks)
     return blocks, pruned
_______________________________________________
Mercurial-devel mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[PATCH 4 of 4] [RFC] releasenotes: command to manage release notes files

Gregory Szorc
In reply to this post by Gregory Szorc
# HG changeset patch
# User Gregory Szorc <[hidden email]>
# Date 1487210192 28800
#      Wed Feb 15 17:56:32 2017 -0800
# Node ID 9ab96e9d989c0b722ce0eedcb357e449c7bd5553
# Parent  9ab3ad4934aedc649d28f74dd70207c6c6e88596
[RFC] releasenotes: command to manage release notes files

Per discussion on the mailing list, we want better release notes
for Mercurial.

This patch introduces an extension that provides a command for
producing release notes files. Functionality is implemented
as an extension because it could be useful outside of the
Mercurial project and because there is some code (like rst
parsing) that already exists in Mercurial and it doesn't make
sense to reinvent the wheel.

The general idea with the extension is that changeset authors
declare release notes in commit messages using rst directives.
Periodically (such as at publishing or release time), a project
maintainer runs `hg releasenotes` to extract release notes
fragments from commit messages and format them to an auto-generated
release notes file. More details are explained inline in docstrings.

There are several things that need addressed before this is ready
for prime time:

* Moar tests
* Interactive merge mode
* Implement similarity detection for individual notes items
* Support customizing section names/titles
* Parsing improvements for bullet lists and paragraphs
* Document which rst primitives can be parsed
* Retain arbitrary content (e.g. header section/paragraphs)
  from existing release notes file
* Better error messages (line numbers, hints, etc)

diff --git a/hgext/releasenotes.py b/hgext/releasenotes.py
new file mode 100644
--- /dev/null
+++ b/hgext/releasenotes.py
@@ -0,0 +1,435 @@
+# Copyright 2017-present Gregory Szorc <[hidden email]>
+#
+# This software may be used and distributed according to the terms of the
+# GNU General Public License version 2 or any later version.
+
+"""generate release notes from commit messages (EXPERIMENTAL)
+
+It is common to maintain files detailing changes in a project between
+releases. Maintaining these files can be difficult and time consuming.
+The :hg:`releasenotes` command provided by this extension makes the
+process simpler by automating it.
+"""
+
+from __future__ import absolute_import
+
+import errno
+import re
+import sys
+import textwrap
+
+from mercurial.i18n import _
+from mercurial import (
+    cmdutil,
+    error,
+    minirst,
+    scmutil,
+)
+
+cmdtable = {}
+command = cmdutil.command(cmdtable)
+
+# Note for extension authors: ONLY specify testedwith = 'ships-with-hg-core' for
+# extensions which SHIP WITH MERCURIAL. Non-mainline extensions should
+# be specifying the version(s) of Mercurial they are tested with, or
+# leave the attribute unspecified.
+testedwith = 'ships-with-hg-core'
+
+DEFAULT_SECTIONS = [
+    ('feature', _('New Features')),
+    ('bc', _('Backwards Compatibility Changes')),
+    ('fix', _('Bug Fixes')),
+    ('perf', _('Performance Improvements')),
+    ('api', _('API Changes')),
+]
+
+RE_DIRECTIVE = re.compile('^\.\. ([a-zA-Z0-9_]+)::\s*([^$]+)?$')
+
+BULLET_SECTION = _('Other Changes')
+
+class parsedreleasenotes(object):
+    def __init__(self):
+        self.sections = {}
+
+    def __contains__(self, section):
+        return section in self.sections
+
+    def __iter__(self):
+        return iter(sorted(self.sections))
+
+    def addtitleditem(self, section, title, paragraphs):
+        """Add a titled release note entry."""
+        self.sections.setdefault(section, ([], []))
+        self.sections[section][0].append((title, paragraphs))
+
+    def addnontitleditem(self, section, paragraphs):
+        """Adds a non-titled release note entry.
+
+        Will be rendered as a bullet point.
+        """
+        self.sections.setdefault(section, ([], []))
+        self.sections[section][1].append(paragraphs)
+
+    def titledforsection(self, section):
+        """Returns titled entries in a section.
+
+        Returns a list of (title, paragraphs) tuples describing sub-sections.
+        """
+        return self.sections.get(section, ([], []))[0]
+
+    def nontitledforsection(self, section):
+        """Returns non-titled, bulleted paragraphs in a section."""
+        return self.sections.get(section, ([], []))[1]
+
+    def hastitledinsection(self, section, title):
+        return any(t[0] == title for t in self.titledforsection(section))
+
+    def merge(self, ui, other):
+        """Merge another instance into this one.
+
+        This is used to combine multiple sources of release notes together.
+        """
+        for section in other:
+            for title, paragraphs in other.titledforsection(section):
+                if self.hastitledinsection(section, title):
+                    # TODO prompt for resolution if different and running in
+                    # interactive mode.
+                    ui.write('%s already exists in %s section; ignoring\n' %
+                             (title, section))
+                    continue
+
+                # TODO perform similarity comparison and try to match against
+                # existing.
+                self.addtitleditem(section, title, paragraphs)
+
+            for paragraphs in other.nontitledforsection(section):
+                if paragraphs in self.nontitledforsection(section):
+                    continue
+
+                # TODO perform similarily comparison and try to match against
+                # existing.
+                self.addnontitleditem(section, paragraphs)
+
+class releasenotessections(object):
+    def __init__(self, ui):
+        # TODO support defining custom sections from config.
+        self._sections = list(DEFAULT_SECTIONS)
+
+    def __iter__(self):
+        return iter(self._sections)
+
+    def names(self):
+        return [t[0] for t in self._sections]
+
+    def sectionfromtitle(self, title):
+        for name, value in self._sections:
+            if value == title:
+                return name
+
+        return None
+
+def parsenotesfromrevisions(repo, directives, revs):
+    notes = parsedreleasenotes()
+
+    for rev in revs:
+        ctx = repo[rev]
+
+        blocks, pruned = minirst.parse(ctx.description(),
+                                       admonitions=directives)
+
+        for i, block in enumerate(blocks):
+            if block['type'] != 'admonition':
+                continue
+
+            directive = block['admonitiontitle']
+            title = block['lines'][0].strip() if block['lines'] else None
+
+            if i + 1 == len(blocks):
+                raise error.Abort(_('release notes directive %s lacks content')
+                                  % directive)
+
+            # Now search ahead and find all paragraphs attached to this
+            # admonition.
+            paragraphs = []
+            for j in range(i + 1, len(blocks)):
+                pblock = blocks[j]
+
+                # Margin blocks may appear between paragraphs. Ignore them.
+                if pblock['type'] == 'margin':
+                    continue
+
+                if pblock['type'] != 'paragraph':
+                    raise error.Abort(_('unexpected block in release notes '
+                                        'directive %s') % directive)
+
+                if pblock['indent'] > 0:
+                    paragraphs.append(pblock['lines'])
+                else:
+                    break
+
+            # TODO consider using title as paragraph for more concise notes.
+            if not paragraphs:
+                raise error.Abort(_('could not find content for release note '
+                                    '%s') % directive)
+
+            if title:
+                notes.addtitleditem(directive, title, paragraphs)
+            else:
+                notes.addnontitleditem(directive, paragraphs)
+
+    return notes
+
+
+def parsereleasenotesfile(sections, text):
+    """Parse text content containing generated release notes."""
+    notes = parsedreleasenotes()
+
+    blocks = minirst.parse(text)[0]
+
+    def gatherparagraphs(offset):
+        paragraphs = []
+
+        for i in range(offset + 1, len(blocks)):
+            block = blocks[i]
+
+            if block['type'] == 'margin':
+                continue
+            elif block['type'] == 'section':
+                break
+            elif block['type'] == 'bullet':
+                if block['indent'] != 0:
+                    raise error.Abort(_('indented bullet lists not supported'))
+
+                lines = [l[1:].strip() for l in block['lines']]
+                paragraphs.append(lines)
+                continue
+            elif block['type'] != 'paragraph':
+                raise error.Abort(_('unexpected block type in release notes: '
+                                    '%s') % block['type'])
+
+            paragraphs.append(block['lines'])
+
+        return paragraphs
+
+    currentsection = None
+    for i, block in enumerate(blocks):
+        if block['type'] != 'section':
+            continue
+
+        title = block['lines'][0]
+
+        # TODO the parsing around paragraphs and bullet points needs some
+        # work.
+
+        # Main section.
+        if block['underline'] == '=':
+            name = sections.sectionfromtitle(title)
+            if not name:
+                raise error.Abort(_('unknown release notes section: %s') %
+                                  title)
+
+            currentsection = name
+            paragraphs = gatherparagraphs(i)
+            if paragraphs:
+                notes.addnontitleditem(currentsection, paragraphs)
+
+        # Sub-section.
+        elif block['underline'] == '-':
+            paragraphs = gatherparagraphs(i)
+
+            if title == BULLET_SECTION:
+                notes.addnontitleditem(currentsection, paragraphs)
+            else:
+                notes.addtitleditem(currentsection, title, paragraphs)
+        else:
+            raise error.Abort(_('unsupported section type for %s') % title)
+
+    return notes
+
+
+def serializenotes(sections, notes):
+    """Serialize release notes from parsed fragments and notes.
+
+    This function essentially takes the output of ``parsenotesfromrevisions()``
+    and ``parserelnotesfile()`` and produces output combining the 2.
+    """
+    lines = []
+
+    for sectionname, sectiontitle in sections:
+        if sectionname not in notes:
+            continue
+
+        lines.append(sectiontitle)
+        lines.append('=' * len(sectiontitle))
+        lines.append('')
+
+        # First pass to emit sub-sections.
+        for title, paragraphs in notes.titledforsection(sectionname):
+            lines.append(title)
+            lines.append('-' * len(title))
+            lines.append('')
+
+            wrapper = textwrap.TextWrapper(width=78)
+            for i, para in enumerate(paragraphs):
+                if i:
+                    lines.append('')
+                lines.extend(wrapper.wrap(' '.join(para)))
+
+            lines.append('')
+
+        # Second pass to emit bullet list items.
+
+        # If the section has titled and non-titled items, we can't
+        # simply emit the bullet list because it would appear to come
+        # from the last title/section. So, we emit a new sub-section
+        # for the non-titled items.
+        nontitled = notes.nontitledforsection(sectionname)
+        if notes.titledforsection(sectionname) and nontitled:
+            # TODO make configurable.
+            lines.append(BULLET_SECTION)
+            lines.append('-' * len(BULLET_SECTION))
+            lines.append('')
+
+        for paragraphs in nontitled:
+            wrapper = textwrap.TextWrapper(initial_indent='* ',
+                                           subsequent_indent='  ',
+                                           width=78)
+            lines.extend(wrapper.wrap(' '.join(paragraphs[0])))
+
+            wrapper = textwrap.TextWrapper(initial_indent='  ',
+                                           subsequent_indent='  ',
+                                           width=78)
+            for para in paragraphs[1:]:
+                lines.append('')
+                lines.extend(wrapper.wrap(' '.join(para)))
+
+            lines.append('')
+
+    if lines[-1]:
+        lines.append('')
+
+    return '\n'.join(lines)
+
+
+@command('releasenotes',
+    [('r', 'rev', '', _('revisions to process for release notes'), _('REV'))],
+    _('[-r REV] FILE'))
+def releasenotes(ui, repo, file_, rev=None):
+    """parse release notes from commit messages into an output file
+
+    Given an output file and set of revisions, this command will parse commit
+    messages for release notes then add them to the output file.
+
+    Release notes are defined in commit messages as ReStructuredText
+    directives. These have the form::
+
+       .. directive:: title
+
+          content
+
+    Each ``directive`` maps to an output section in a generated release notes
+    file, which itself is ReStructuredText. For example, the ``.. feature::``
+    directive would map to a ``New Features`` section.
+
+    Release note directives can be either short-form or long-form. In short-
+    form, ``title`` is omitted and the release note is rendered as a bullet
+    list. In long form, a sub-section with the title ``title`` is added to the
+    section.
+
+    The ``FILE`` argument controls the output file to write gathered release
+    notes to. The format of the file is::
+
+       Section 1
+       =========
+
+       ...
+
+       Section 2
+       =========
+
+       ...
+
+    Only sections with defined release notes are emitted.
+
+    If a section only has short-form notes, it will consist of bullet list::
+
+       Section
+       =======
+
+       * Release note 1
+       * Release note 2
+
+    If a section has long-form notes, sub-sections will be emitted::
+
+       Section
+       =======
+
+       Note 1 Title
+       ------------
+
+       Description of the first long-form note.
+
+       Note 2 Title
+       ------------
+
+       Description of the second long-form note.
+
+    If the ``FILE`` argument points to an existing file, that file will be
+    parsed for release notes having the format that would be generated by this
+    command. The notes from the processed commit messages will be *merged*
+    into this parsed set.
+
+    During release notes merging:
+
+    * Duplicate items are automatically ignored
+    * Items that are different are automatically ignored if the similarity is
+      greater than a threshold.
+
+    This means that the release notes file can be updated independently from
+    this command and changes should not be lost when running this command on
+    that file. A particular use case for this is to tweak the wording of a
+    release note after it has been added to the release notes file.
+    """
+    sections = releasenotessections(ui)
+
+    revs = scmutil.revrange(repo, [rev or 'not public()'])
+    incoming = parsenotesfromrevisions(repo, sections.names(), revs)
+
+    try:
+        with open(file_, 'rb') as fh:
+            notes = parsereleasenotesfile(sections, fh.read())
+    except IOError as e:
+        if e.errno != errno.ENOENT:
+            raise
+
+        notes = parsedreleasenotes()
+
+    notes.merge(ui, incoming)
+
+    with open(file_, 'wb') as fh:
+        fh.write(serializenotes(sections, notes))
+
+@command('debugparsereleasenotes', norepo=True)
+def debugparsereleasenotes(ui, path):
+    """parse release notes and print resulting data structure"""
+    if path == '-':
+        text = sys.stdin.read()
+    else:
+        with open(path, 'rb') as fh:
+            text = fh.read()
+
+    sections = releasenotessections(ui)
+
+    notes = parsereleasenotesfile(sections, text)
+
+    for section in notes:
+        ui.write('section: %s\n' % section)
+        for title, paragraphs in notes.titledforsection(section):
+            ui.write('  subsection: %s\n' % title)
+            for para in paragraphs:
+                ui.write('    paragraph: %s\n' % ' '.join(para))
+
+        for paragraphs in notes.nontitledforsection(section):
+            ui.write('  bullet point:\n')
+            for para in paragraphs:
+                ui.write('    paragraph: %s\n' % ' '.join(para))
diff --git a/tests/test-releasenotes-formatting.t b/tests/test-releasenotes-formatting.t
new file mode 100644
--- /dev/null
+++ b/tests/test-releasenotes-formatting.t
@@ -0,0 +1,256 @@
+  $ cat >> $HGRCPATH << EOF
+  > [extensions]
+  > releasenotes=
+  > EOF
+
+  $ hg init simple-repo
+  $ cd simple-repo
+
+A fix with a single line results in a bullet point in the appropriate section
+
+  $ touch fix1
+  $ hg -q commit -A -l - << EOF
+  > single line fix
+  >
+  > .. fix::
+  >
+  >    Simple fix with a single line content entry.
+  > EOF
+
+  $ hg releasenotes -r . $TESTTMP/relnotes-single-line
+
+  $ cat $TESTTMP/relnotes-single-line
+  Bug Fixes
+  =========
+  
+  * Simple fix with a single line content entry.
+
+A fix with multiple lines is handled correctly
+
+  $ touch fix2
+  $ hg -q commit -A -l - << EOF
+  > multi line fix
+  >
+  > .. fix::
+  >
+  >    First line of fix entry.
+  >    A line after it without a space.
+  >
+  >    A new paragraph in the fix entry. And this is a really long line. It goes on for a while.
+  >    And it wraps around to a new paragraph.
+  > EOF
+
+  $ hg releasenotes -r . $TESTTMP/relnotes-multi-line
+  $ cat $TESTTMP/relnotes-multi-line
+  Bug Fixes
+  =========
+  
+  * First line of fix entry. A line after it without a space.
+  
+    A new paragraph in the fix entry. And this is a really long line. It goes on
+    for a while. And it wraps around to a new paragraph.
+
+A release note with a title results in a sub-section being written
+
+  $ touch fix3
+  $ hg -q commit -A -l - << EOF
+  > fix with title
+  >
+  > .. fix:: Fix Title
+  >
+  >    First line of fix with title.
+  >
+  >    Another paragraph of fix with title. But this is a paragraph
+  >    with multiple lines.
+  > EOF
+
+  $ hg releasenotes -r . $TESTTMP/relnotes-fix-with-title
+  $ cat $TESTTMP/relnotes-fix-with-title
+  Bug Fixes
+  =========
+  
+  Fix Title
+  ---------
+  
+  First line of fix with title.
+  
+  Another paragraph of fix with title. But this is a paragraph with multiple
+  lines.
+
+  $ cd ..
+
+Formatting of multiple bullet points works
+
+  $ hg init multiple-bullets
+  $ cd multiple-bullets
+  $ touch fix1
+  $ hg -q commit -A -l - << EOF
+  > commit 1
+  >
+  > .. fix::
+  >
+  >    first fix
+  > EOF
+
+  $ touch fix2
+  $ hg -q commit -A -l - << EOF
+  > commit 2
+  >
+  > .. fix::
+  >
+  >    second fix
+  >
+  >    Second paragraph of second fix.
+  > EOF
+
+  $ touch fix3
+  $ hg -q commit -A -l - << EOF
+  > commit 3
+  >
+  > .. fix::
+  >
+  >    third fix
+  > EOF
+
+  $ hg releasenotes -r 'all()' $TESTTMP/relnotes-multiple-bullets
+  $ cat $TESTTMP/relnotes-multiple-bullets
+  Bug Fixes
+  =========
+  
+  * first fix
+  
+  * second fix
+  
+    Second paragraph of second fix.
+  
+  * third fix
+
+  $ cd ..
+
+Formatting of multiple sections works
+
+  $ hg init multiple-sections
+  $ cd multiple-sections
+  $ touch fix1
+  $ hg -q commit -A -l - << EOF
+  > commit 1
+  >
+  > .. fix::
+  >
+  >    first fix
+  > EOF
+
+  $ touch feature1
+  $ hg -q commit -A -l - << EOF
+  > commit 2
+  >
+  > .. feature::
+  >
+  >    description of the new feature
+  > EOF
+
+  $ touch fix2
+  $ hg -q commit -A -l - << EOF
+  > commit 3
+  >
+  > .. fix::
+  >
+  >    second fix
+  > EOF
+
+  $ hg releasenotes -r 'all()' $TESTTMP/relnotes-multiple-sections
+  $ cat $TESTTMP/relnotes-multiple-sections
+  New Features
+  ============
+  
+  * description of the new feature
+  
+  Bug Fixes
+  =========
+  
+  * first fix
+  
+  * second fix
+
+  $ cd ..
+
+Section with subsections and bullets
+
+  $ hg init multiple-subsections
+  $ cd multiple-subsections
+
+  $ touch fix1
+  $ hg -q commit -A -l - << EOF
+  > commit 1
+  >
+  > .. fix:: Title of First Fix
+  >
+  >    First paragraph of first fix.
+  >
+  >    Second paragraph of first fix.
+  > EOF
+
+  $ touch fix2
+  $ hg -q commit -A -l - << EOF
+  > commit 2
+  >
+  > .. fix:: Title of Second Fix
+  >
+  >    First paragraph of second fix.
+  >
+  >    Second paragraph of second fix.
+  > EOF
+
+  $ hg releasenotes -r 'all()' $TESTTMP/relnotes-multiple-subsections
+  $ cat $TESTTMP/relnotes-multiple-subsections
+  Bug Fixes
+  =========
+  
+  Title of First Fix
+  ------------------
+  
+  First paragraph of first fix.
+  
+  Second paragraph of first fix.
+  
+  Title of Second Fix
+  -------------------
+  
+  First paragraph of second fix.
+  
+  Second paragraph of second fix.
+
+Now add bullet points to sections having sub-sections
+
+  $ touch fix3
+  $ hg -q commit -A -l - << EOF
+  > commit 3
+  >
+  > .. fix::
+  >
+  >    Short summary of fix 3
+  > EOF
+
+  $ hg releasenotes -r 'all()' $TESTTMP/relnotes-multiple-subsections-with-bullets
+  $ cat $TESTTMP/relnotes-multiple-subsections-with-bullets
+  Bug Fixes
+  =========
+  
+  Title of First Fix
+  ------------------
+  
+  First paragraph of first fix.
+  
+  Second paragraph of first fix.
+  
+  Title of Second Fix
+  -------------------
+  
+  First paragraph of second fix.
+  
+  Second paragraph of second fix.
+  
+  Other Changes
+  -------------
+  
+  * Short summary of fix 3
diff --git a/tests/test-releasenotes-merging.t b/tests/test-releasenotes-merging.t
new file mode 100644
--- /dev/null
+++ b/tests/test-releasenotes-merging.t
@@ -0,0 +1,112 @@
+  $ cat >> $HGRCPATH << EOF
+  > [extensions]
+  > releasenotes=
+  > EOF
+
+  $ hg init simple-repo
+  $ cd simple-repo
+
+A fix directive from commit message is added to release notes
+
+  $ touch fix1
+  $ hg -q commit -A -l - << EOF
+  > commit 1
+  >
+  > .. fix::
+  >
+  >    Fix from commit message.
+  > EOF
+
+  $ cat >> $TESTTMP/single-fix-bullet << EOF
+  > Bug Fixes
+  > =========
+  >
+  > * Fix from release notes.
+  > EOF
+
+  $ hg releasenotes -r . $TESTTMP/single-fix-bullet
+
+  $ cat $TESTTMP/single-fix-bullet
+  Bug Fixes
+  =========
+  
+  * Fix from release notes.
+  
+  * Fix from commit message.
+
+Processing again will no-op
+TODO this is buggy
+
+  $ hg releasenotes -r . $TESTTMP/single-fix-bullet
+
+  $ cat $TESTTMP/single-fix-bullet
+  Bug Fixes
+  =========
+  
+  * Fix from release notes.
+  
+    Fix from commit message.
+  
+  * Fix from commit message.
+
+  $ cd ..
+
+Sections are unioned
+
+  $ hg init subsections
+  $ cd subsections
+  $ touch fix1
+  $ hg -q commit -A -l - << EOF
+  > Commit 1
+  >
+  > .. feature:: Commit Message Feature
+  >
+  >    This describes a feature from a commit message.
+  > EOF
+
+  $ cat >> $TESTTMP/single-feature-section << EOF
+  > New Features
+  > ============
+  >
+  > Notes Feature
+  > -------------
+  >
+  > This describes a feature from a release notes file.
+  > EOF
+
+  $ hg releasenotes -r . $TESTTMP/single-feature-section
+
+  $ cat $TESTTMP/single-feature-section
+  New Features
+  ============
+  
+  Notes Feature
+  -------------
+  
+  This describes a feature from a release notes file.
+  
+  Commit Message Feature
+  ----------------------
+  
+  This describes a feature from a commit message.
+
+Doing it again won't add another section
+
+  $ hg releasenotes -r . $TESTTMP/single-feature-section
+  Commit Message Feature already exists in feature section; ignoring
+
+  $ cat $TESTTMP/single-feature-section
+  New Features
+  ============
+  
+  Notes Feature
+  -------------
+  
+  This describes a feature from a release notes file.
+  
+  Commit Message Feature
+  ----------------------
+  
+  This describes a feature from a commit message.
+
+  $ cd ..
diff --git a/tests/test-releasenotes-parsing.t b/tests/test-releasenotes-parsing.t
new file mode 100644
--- /dev/null
+++ b/tests/test-releasenotes-parsing.t
@@ -0,0 +1,170 @@
+  $ cat >> $HGRCPATH << EOF
+  > [extensions]
+  > releasenotes=
+  > EOF
+
+Bullet point with a single item spanning a single line
+
+  $ hg debugparsereleasenotes - << EOF
+  > New Features
+  > ============
+  >
+  > * Bullet point item with a single line
+  > EOF
+  section: feature
+    bullet point:
+      paragraph: Bullet point item with a single line
+
+Bullet point that spans multiple lines.
+
+  $ hg debugparsereleasenotes - << EOF
+  > New Features
+  > ============
+  >
+  > * Bullet point with a paragraph
+  >   that spans multiple lines.
+  > EOF
+  section: feature
+    bullet point:
+      paragraph: Bullet point with a paragraph that spans multiple lines.
+
+  $ hg debugparsereleasenotes - << EOF
+  > New Features
+  > ============
+  >
+  > * Bullet point with a paragraph
+  >   that spans multiple lines.
+  >
+  >   And has an empty line between lines too.
+  >   With a line cuddling that.
+  > EOF
+  section: feature
+    bullet point:
+      paragraph: Bullet point with a paragraph that spans multiple lines.
+      paragraph: And has an empty line between lines too. With a line cuddling that.
+
+
+Multiple bullet points. With some entries being multiple lines.
+
+  $ hg debugparsereleasenotes - << EOF
+  > New Features
+  > ============
+  >
+  > * First bullet point. It has a single line.
+  >
+  > * Second bullet point.
+  >   It consists of multiple lines.
+  >
+  > * Third bullet point. It has a single line.
+  > EOF
+  section: feature
+    bullet point:
+      paragraph: First bullet point. It has a single line.
+      paragraph: Second bullet point. It consists of multiple lines.
+      paragraph: Third bullet point. It has a single line.
+
+Bullet point without newline between items
+
+  $ hg debugparsereleasenotes - << EOF
+  > New Features
+  > ============
+  >
+  > * First bullet point
+  > * Second bullet point
+  >   And it has multiple lines
+  > * Third bullet point
+  > * Fourth bullet point
+  > EOF
+  section: feature
+    bullet point:
+      paragraph: First bullet point
+      paragraph: Second bullet point And it has multiple lines
+      paragraph: Third bullet point
+      paragraph: Fourth bullet point
+
+Sub-section contents are read
+
+  $ hg debugparsereleasenotes - << EOF
+  > New Features
+  > ============
+  >
+  > First Feature
+  > -------------
+  >
+  > This is the first new feature that was implemented.
+  >
+  > And a second paragraph about it.
+  >
+  > Second Feature
+  > --------------
+  >
+  > This is the second new feature that was implemented.
+  >
+  > Paragraph two.
+  >
+  > Paragraph three.
+  > EOF
+  section: feature
+    subsection: First Feature
+      paragraph: This is the first new feature that was implemented.
+      paragraph: And a second paragraph about it.
+    subsection: Second Feature
+      paragraph: This is the second new feature that was implemented.
+      paragraph: Paragraph two.
+      paragraph: Paragraph three.
+
+Multiple sections are read
+
+  $ hg debugparsereleasenotes - << EOF
+  > New Features
+  > ============
+  >
+  > * Feature 1
+  > * Feature 2
+  >
+  > Bug Fixes
+  > =========
+  >
+  > * Fix 1
+  > * Fix 2
+  > EOF
+  section: feature
+    bullet point:
+      paragraph: Feature 1
+      paragraph: Feature 2
+  section: fix
+    bullet point:
+      paragraph: Fix 1
+      paragraph: Fix 2
+
+Mixed sub-sections and bullet list
+
+  $ hg debugparsereleasenotes - << EOF
+  > New Features
+  > ============
+  >
+  > Feature 1
+  > ---------
+  >
+  > Some words about the first feature.
+  >
+  > Feature 2
+  > ---------
+  >
+  > Some words about the second feature.
+  > That span multiple lines.
+  >
+  > Other Changes
+  > -------------
+  >
+  > * Bullet item 1
+  > * Bullet item 2
+  > EOF
+  section: feature
+    subsection: Feature 1
+      paragraph: Some words about the first feature.
+    subsection: Feature 2
+      paragraph: Some words about the second feature. That span multiple lines.
+    bullet point:
+      paragraph: Bullet item 1
+      paragraph: Bullet item 2
_______________________________________________
Mercurial-devel mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [PATCH 4 of 4] [RFC] releasenotes: command to manage release notes files

Gregory Szorc
On Wed, Feb 15, 2017 at 5:59 PM, Gregory Szorc <[hidden email]> wrote:
# HG changeset patch
# User Gregory Szorc <[hidden email]>
# Date 1487210192 28800
#      Wed Feb 15 17:56:32 2017 -0800
# Node ID 9ab96e9d989c0b722ce0eedcb357e449c7bd5553
# Parent  9ab3ad4934aedc649d28f74dd70207c6c6e88596
[RFC] releasenotes: command to manage release notes files

I think this extension/command addresses the feedback and concerns people had with the previous RFC approach (which was a simple script). The approach in this commit is much more robust. It still isn't complete. But at least you should be able to see where I'm going and how I intend to solve problems like preserving changes to the release notes file made outside of commit messages.

The previous 3 minirst patches are needed to make this extension work. The bullet list change can stand on its own and can be queued if a reviewer finds it appropriate. The 2 for admonitions parsing are only needed if we want to move forward with the approach in this extension.

I'll be AFK for the next several days (possibly a week or two) and won't have time to iterate on this. If someone else wants to pick it up and try to get it queued, go for it!
 

Per discussion on the mailing list, we want better release notes
for Mercurial.

This patch introduces an extension that provides a command for
producing release notes files. Functionality is implemented
as an extension because it could be useful outside of the
Mercurial project and because there is some code (like rst
parsing) that already exists in Mercurial and it doesn't make
sense to reinvent the wheel.

The general idea with the extension is that changeset authors
declare release notes in commit messages using rst directives.
Periodically (such as at publishing or release time), a project
maintainer runs `hg releasenotes` to extract release notes
fragments from commit messages and format them to an auto-generated
release notes file. More details are explained inline in docstrings.

There are several things that need addressed before this is ready
for prime time:

* Moar tests
* Interactive merge mode
* Implement similarity detection for individual notes items
* Support customizing section names/titles
* Parsing improvements for bullet lists and paragraphs
* Document which rst primitives can be parsed
* Retain arbitrary content (e.g. header section/paragraphs)
  from existing release notes file
* Better error messages (line numbers, hints, etc)

diff --git a/hgext/releasenotes.py b/hgext/releasenotes.py
new file mode 100644
--- /dev/null
+++ b/hgext/releasenotes.py
@@ -0,0 +1,435 @@
+# Copyright 2017-present Gregory Szorc <[hidden email]>
+#
+# This software may be used and distributed according to the terms of the
+# GNU General Public License version 2 or any later version.
+
+"""generate release notes from commit messages (EXPERIMENTAL)
+
+It is common to maintain files detailing changes in a project between
+releases. Maintaining these files can be difficult and time consuming.
+The :hg:`releasenotes` command provided by this extension makes the
+process simpler by automating it.
+"""
+
+from __future__ import absolute_import
+
+import errno
+import re
+import sys
+import textwrap
+
+from mercurial.i18n import _
+from mercurial import (
+    cmdutil,
+    error,
+    minirst,
+    scmutil,
+)
+
+cmdtable = {}
+command = cmdutil.command(cmdtable)
+
+# Note for extension authors: ONLY specify testedwith = 'ships-with-hg-core' for
+# extensions which SHIP WITH MERCURIAL. Non-mainline extensions should
+# be specifying the version(s) of Mercurial they are tested with, or
+# leave the attribute unspecified.
+testedwith = 'ships-with-hg-core'
+
+DEFAULT_SECTIONS = [
+    ('feature', _('New Features')),
+    ('bc', _('Backwards Compatibility Changes')),
+    ('fix', _('Bug Fixes')),
+    ('perf', _('Performance Improvements')),
+    ('api', _('API Changes')),
+]
+
+RE_DIRECTIVE = re.compile('^\.\. ([a-zA-Z0-9_]+)::\s*([^$]+)?$')
+
+BULLET_SECTION = _('Other Changes')
+
+class parsedreleasenotes(object):
+    def __init__(self):
+        self.sections = {}
+
+    def __contains__(self, section):
+        return section in self.sections
+
+    def __iter__(self):
+        return iter(sorted(self.sections))
+
+    def addtitleditem(self, section, title, paragraphs):
+        """Add a titled release note entry."""
+        self.sections.setdefault(section, ([], []))
+        self.sections[section][0].append((title, paragraphs))
+
+    def addnontitleditem(self, section, paragraphs):
+        """Adds a non-titled release note entry.
+
+        Will be rendered as a bullet point.
+        """
+        self.sections.setdefault(section, ([], []))
+        self.sections[section][1].append(paragraphs)
+
+    def titledforsection(self, section):
+        """Returns titled entries in a section.
+
+        Returns a list of (title, paragraphs) tuples describing sub-sections.
+        """
+        return self.sections.get(section, ([], []))[0]
+
+    def nontitledforsection(self, section):
+        """Returns non-titled, bulleted paragraphs in a section."""
+        return self.sections.get(section, ([], []))[1]
+
+    def hastitledinsection(self, section, title):
+        return any(t[0] == title for t in self.titledforsection(section))
+
+    def merge(self, ui, other):
+        """Merge another instance into this one.
+
+        This is used to combine multiple sources of release notes together.
+        """
+        for section in other:
+            for title, paragraphs in other.titledforsection(section):
+                if self.hastitledinsection(section, title):
+                    # TODO prompt for resolution if different and running in
+                    # interactive mode.
+                    ui.write('%s already exists in %s section; ignoring\n' %
+                             (title, section))
+                    continue
+
+                # TODO perform similarity comparison and try to match against
+                # existing.
+                self.addtitleditem(section, title, paragraphs)
+
+            for paragraphs in other.nontitledforsection(section):
+                if paragraphs in self.nontitledforsection(section):
+                    continue
+
+                # TODO perform similarily comparison and try to match against
+                # existing.
+                self.addnontitleditem(section, paragraphs)
+
+class releasenotessections(object):
+    def __init__(self, ui):
+        # TODO support defining custom sections from config.
+        self._sections = list(DEFAULT_SECTIONS)
+
+    def __iter__(self):
+        return iter(self._sections)
+
+    def names(self):
+        return [t[0] for t in self._sections]
+
+    def sectionfromtitle(self, title):
+        for name, value in self._sections:
+            if value == title:
+                return name
+
+        return None
+
+def parsenotesfromrevisions(repo, directives, revs):
+    notes = parsedreleasenotes()
+
+    for rev in revs:
+        ctx = repo[rev]
+
+        blocks, pruned = minirst.parse(ctx.description(),
+                                       admonitions=directives)
+
+        for i, block in enumerate(blocks):
+            if block['type'] != 'admonition':
+                continue
+
+            directive = block['admonitiontitle']
+            title = block['lines'][0].strip() if block['lines'] else None
+
+            if i + 1 == len(blocks):
+                raise error.Abort(_('release notes directive %s lacks content')
+                                  % directive)
+
+            # Now search ahead and find all paragraphs attached to this
+            # admonition.
+            paragraphs = []
+            for j in range(i + 1, len(blocks)):
+                pblock = blocks[j]
+
+                # Margin blocks may appear between paragraphs. Ignore them.
+                if pblock['type'] == 'margin':
+                    continue
+
+                if pblock['type'] != 'paragraph':
+                    raise error.Abort(_('unexpected block in release notes '
+                                        'directive %s') % directive)
+
+                if pblock['indent'] > 0:
+                    paragraphs.append(pblock['lines'])
+                else:
+                    break
+
+            # TODO consider using title as paragraph for more concise notes.
+            if not paragraphs:
+                raise error.Abort(_('could not find content for release note '
+                                    '%s') % directive)
+
+            if title:
+                notes.addtitleditem(directive, title, paragraphs)
+            else:
+                notes.addnontitleditem(directive, paragraphs)
+
+    return notes
+
+
+def parsereleasenotesfile(sections, text):
+    """Parse text content containing generated release notes."""
+    notes = parsedreleasenotes()
+
+    blocks = minirst.parse(text)[0]
+
+    def gatherparagraphs(offset):
+        paragraphs = []
+
+        for i in range(offset + 1, len(blocks)):
+            block = blocks[i]
+
+            if block['type'] == 'margin':
+                continue
+            elif block['type'] == 'section':
+                break
+            elif block['type'] == 'bullet':
+                if block['indent'] != 0:
+                    raise error.Abort(_('indented bullet lists not supported'))
+
+                lines = [l[1:].strip() for l in block['lines']]
+                paragraphs.append(lines)
+                continue
+            elif block['type'] != 'paragraph':
+                raise error.Abort(_('unexpected block type in release notes: '
+                                    '%s') % block['type'])
+
+            paragraphs.append(block['lines'])
+
+        return paragraphs
+
+    currentsection = None
+    for i, block in enumerate(blocks):
+        if block['type'] != 'section':
+            continue
+
+        title = block['lines'][0]
+
+        # TODO the parsing around paragraphs and bullet points needs some
+        # work.
+
+        # Main section.
+        if block['underline'] == '=':
+            name = sections.sectionfromtitle(title)
+            if not name:
+                raise error.Abort(_('unknown release notes section: %s') %
+                                  title)
+
+            currentsection = name
+            paragraphs = gatherparagraphs(i)
+            if paragraphs:
+                notes.addnontitleditem(currentsection, paragraphs)
+
+        # Sub-section.
+        elif block['underline'] == '-':
+            paragraphs = gatherparagraphs(i)
+
+            if title == BULLET_SECTION:
+                notes.addnontitleditem(currentsection, paragraphs)
+            else:
+                notes.addtitleditem(currentsection, title, paragraphs)
+        else:
+            raise error.Abort(_('unsupported section type for %s') % title)
+
+    return notes
+
+
+def serializenotes(sections, notes):
+    """Serialize release notes from parsed fragments and notes.
+
+    This function essentially takes the output of ``parsenotesfromrevisions()``
+    and ``parserelnotesfile()`` and produces output combining the 2.
+    """
+    lines = []
+
+    for sectionname, sectiontitle in sections:
+        if sectionname not in notes:
+            continue
+
+        lines.append(sectiontitle)
+        lines.append('=' * len(sectiontitle))
+        lines.append('')
+
+        # First pass to emit sub-sections.
+        for title, paragraphs in notes.titledforsection(sectionname):
+            lines.append(title)
+            lines.append('-' * len(title))
+            lines.append('')
+
+            wrapper = textwrap.TextWrapper(width=78)
+            for i, para in enumerate(paragraphs):
+                if i:
+                    lines.append('')
+                lines.extend(wrapper.wrap(' '.join(para)))
+
+            lines.append('')
+
+        # Second pass to emit bullet list items.
+
+        # If the section has titled and non-titled items, we can't
+        # simply emit the bullet list because it would appear to come
+        # from the last title/section. So, we emit a new sub-section
+        # for the non-titled items.
+        nontitled = notes.nontitledforsection(sectionname)
+        if notes.titledforsection(sectionname) and nontitled:
+            # TODO make configurable.
+            lines.append(BULLET_SECTION)
+            lines.append('-' * len(BULLET_SECTION))
+            lines.append('')
+
+        for paragraphs in nontitled:
+            wrapper = textwrap.TextWrapper(initial_indent='* ',
+                                           subsequent_indent='  ',
+                                           width=78)
+            lines.extend(wrapper.wrap(' '.join(paragraphs[0])))
+
+            wrapper = textwrap.TextWrapper(initial_indent='  ',
+                                           subsequent_indent='  ',
+                                           width=78)
+            for para in paragraphs[1:]:
+                lines.append('')
+                lines.extend(wrapper.wrap(' '.join(para)))
+
+            lines.append('')
+
+    if lines[-1]:
+        lines.append('')
+
+    return '\n'.join(lines)
+
+
+@command('releasenotes',
+    [('r', 'rev', '', _('revisions to process for release notes'), _('REV'))],
+    _('[-r REV] FILE'))
+def releasenotes(ui, repo, file_, rev=None):
+    """parse release notes from commit messages into an output file
+
+    Given an output file and set of revisions, this command will parse commit
+    messages for release notes then add them to the output file.
+
+    Release notes are defined in commit messages as ReStructuredText
+    directives. These have the form::
+
+       .. directive:: title
+
+          content
+
+    Each ``directive`` maps to an output section in a generated release notes
+    file, which itself is ReStructuredText. For example, the ``.. feature::``
+    directive would map to a ``New Features`` section.
+
+    Release note directives can be either short-form or long-form. In short-
+    form, ``title`` is omitted and the release note is rendered as a bullet
+    list. In long form, a sub-section with the title ``title`` is added to the
+    section.
+
+    The ``FILE`` argument controls the output file to write gathered release
+    notes to. The format of the file is::
+
+       Section 1
+       =========
+
+       ...
+
+       Section 2
+       =========
+
+       ...
+
+    Only sections with defined release notes are emitted.
+
+    If a section only has short-form notes, it will consist of bullet list::
+
+       Section
+       =======
+
+       * Release note 1
+       * Release note 2
+
+    If a section has long-form notes, sub-sections will be emitted::
+
+       Section
+       =======
+
+       Note 1 Title
+       ------------
+
+       Description of the first long-form note.
+
+       Note 2 Title
+       ------------
+
+       Description of the second long-form note.
+
+    If the ``FILE`` argument points to an existing file, that file will be
+    parsed for release notes having the format that would be generated by this
+    command. The notes from the processed commit messages will be *merged*
+    into this parsed set.
+
+    During release notes merging:
+
+    * Duplicate items are automatically ignored
+    * Items that are different are automatically ignored if the similarity is
+      greater than a threshold.
+
+    This means that the release notes file can be updated independently from
+    this command and changes should not be lost when running this command on
+    that file. A particular use case for this is to tweak the wording of a
+    release note after it has been added to the release notes file.
+    """
+    sections = releasenotessections(ui)
+
+    revs = scmutil.revrange(repo, [rev or 'not public()'])
+    incoming = parsenotesfromrevisions(repo, sections.names(), revs)
+
+    try:
+        with open(file_, 'rb') as fh:
+            notes = parsereleasenotesfile(sections, fh.read())
+    except IOError as e:
+        if e.errno != errno.ENOENT:
+            raise
+
+        notes = parsedreleasenotes()
+
+    notes.merge(ui, incoming)
+
+    with open(file_, 'wb') as fh:
+        fh.write(serializenotes(sections, notes))
+
+@command('debugparsereleasenotes', norepo=True)
+def debugparsereleasenotes(ui, path):
+    """parse release notes and print resulting data structure"""
+    if path == '-':
+        text = sys.stdin.read()
+    else:
+        with open(path, 'rb') as fh:
+            text = fh.read()
+
+    sections = releasenotessections(ui)
+
+    notes = parsereleasenotesfile(sections, text)
+
+    for section in notes:
+        ui.write('section: %s\n' % section)
+        for title, paragraphs in notes.titledforsection(section):
+            ui.write('  subsection: %s\n' % title)
+            for para in paragraphs:
+                ui.write('    paragraph: %s\n' % ' '.join(para))
+
+        for paragraphs in notes.nontitledforsection(section):
+            ui.write('  bullet point:\n')
+            for para in paragraphs:
+                ui.write('    paragraph: %s\n' % ' '.join(para))
diff --git a/tests/test-releasenotes-formatting.t b/tests/test-releasenotes-formatting.t
new file mode 100644
--- /dev/null
+++ b/tests/test-releasenotes-formatting.t
@@ -0,0 +1,256 @@
+  $ cat >> $HGRCPATH << EOF
+  > [extensions]
+  > releasenotes=
+  > EOF
+
+  $ hg init simple-repo
+  $ cd simple-repo
+
+A fix with a single line results in a bullet point in the appropriate section
+
+  $ touch fix1
+  $ hg -q commit -A -l - << EOF
+  > single line fix
+  >
+  > .. fix::
+  >
+  >    Simple fix with a single line content entry.
+  > EOF
+
+  $ hg releasenotes -r . $TESTTMP/relnotes-single-line
+
+  $ cat $TESTTMP/relnotes-single-line
+  Bug Fixes
+  =========
+
+  * Simple fix with a single line content entry.
+
+A fix with multiple lines is handled correctly
+
+  $ touch fix2
+  $ hg -q commit -A -l - << EOF
+  > multi line fix
+  >
+  > .. fix::
+  >
+  >    First line of fix entry.
+  >    A line after it without a space.
+  >
+  >    A new paragraph in the fix entry. And this is a really long line. It goes on for a while.
+  >    And it wraps around to a new paragraph.
+  > EOF
+
+  $ hg releasenotes -r . $TESTTMP/relnotes-multi-line
+  $ cat $TESTTMP/relnotes-multi-line
+  Bug Fixes
+  =========
+
+  * First line of fix entry. A line after it without a space.
+
+    A new paragraph in the fix entry. And this is a really long line. It goes on
+    for a while. And it wraps around to a new paragraph.
+
+A release note with a title results in a sub-section being written
+
+  $ touch fix3
+  $ hg -q commit -A -l - << EOF
+  > fix with title
+  >
+  > .. fix:: Fix Title
+  >
+  >    First line of fix with title.
+  >
+  >    Another paragraph of fix with title. But this is a paragraph
+  >    with multiple lines.
+  > EOF
+
+  $ hg releasenotes -r . $TESTTMP/relnotes-fix-with-title
+  $ cat $TESTTMP/relnotes-fix-with-title
+  Bug Fixes
+  =========
+
+  Fix Title
+  ---------
+
+  First line of fix with title.
+
+  Another paragraph of fix with title. But this is a paragraph with multiple
+  lines.
+
+  $ cd ..
+
+Formatting of multiple bullet points works
+
+  $ hg init multiple-bullets
+  $ cd multiple-bullets
+  $ touch fix1
+  $ hg -q commit -A -l - << EOF
+  > commit 1
+  >
+  > .. fix::
+  >
+  >    first fix
+  > EOF
+
+  $ touch fix2
+  $ hg -q commit -A -l - << EOF
+  > commit 2
+  >
+  > .. fix::
+  >
+  >    second fix
+  >
+  >    Second paragraph of second fix.
+  > EOF
+
+  $ touch fix3
+  $ hg -q commit -A -l - << EOF
+  > commit 3
+  >
+  > .. fix::
+  >
+  >    third fix
+  > EOF
+
+  $ hg releasenotes -r 'all()' $TESTTMP/relnotes-multiple-bullets
+  $ cat $TESTTMP/relnotes-multiple-bullets
+  Bug Fixes
+  =========
+
+  * first fix
+
+  * second fix
+
+    Second paragraph of second fix.
+
+  * third fix
+
+  $ cd ..
+
+Formatting of multiple sections works
+
+  $ hg init multiple-sections
+  $ cd multiple-sections
+  $ touch fix1
+  $ hg -q commit -A -l - << EOF
+  > commit 1
+  >
+  > .. fix::
+  >
+  >    first fix
+  > EOF
+
+  $ touch feature1
+  $ hg -q commit -A -l - << EOF
+  > commit 2
+  >
+  > .. feature::
+  >
+  >    description of the new feature
+  > EOF
+
+  $ touch fix2
+  $ hg -q commit -A -l - << EOF
+  > commit 3
+  >
+  > .. fix::
+  >
+  >    second fix
+  > EOF
+
+  $ hg releasenotes -r 'all()' $TESTTMP/relnotes-multiple-sections
+  $ cat $TESTTMP/relnotes-multiple-sections
+  New Features
+  ============
+
+  * description of the new feature
+
+  Bug Fixes
+  =========
+
+  * first fix
+
+  * second fix
+
+  $ cd ..
+
+Section with subsections and bullets
+
+  $ hg init multiple-subsections
+  $ cd multiple-subsections
+
+  $ touch fix1
+  $ hg -q commit -A -l - << EOF
+  > commit 1
+  >
+  > .. fix:: Title of First Fix
+  >
+  >    First paragraph of first fix.
+  >
+  >    Second paragraph of first fix.
+  > EOF
+
+  $ touch fix2
+  $ hg -q commit -A -l - << EOF
+  > commit 2
+  >
+  > .. fix:: Title of Second Fix
+  >
+  >    First paragraph of second fix.
+  >
+  >    Second paragraph of second fix.
+  > EOF
+
+  $ hg releasenotes -r 'all()' $TESTTMP/relnotes-multiple-subsections
+  $ cat $TESTTMP/relnotes-multiple-subsections
+  Bug Fixes
+  =========
+
+  Title of First Fix
+  ------------------
+
+  First paragraph of first fix.
+
+  Second paragraph of first fix.
+
+  Title of Second Fix
+  -------------------
+
+  First paragraph of second fix.
+
+  Second paragraph of second fix.
+
+Now add bullet points to sections having sub-sections
+
+  $ touch fix3
+  $ hg -q commit -A -l - << EOF
+  > commit 3
+  >
+  > .. fix::
+  >
+  >    Short summary of fix 3
+  > EOF
+
+  $ hg releasenotes -r 'all()' $TESTTMP/relnotes-multiple-subsections-with-bullets
+  $ cat $TESTTMP/relnotes-multiple-subsections-with-bullets
+  Bug Fixes
+  =========
+
+  Title of First Fix
+  ------------------
+
+  First paragraph of first fix.
+
+  Second paragraph of first fix.
+
+  Title of Second Fix
+  -------------------
+
+  First paragraph of second fix.
+
+  Second paragraph of second fix.
+
+  Other Changes
+  -------------
+
+  * Short summary of fix 3
diff --git a/tests/test-releasenotes-merging.t b/tests/test-releasenotes-merging.t
new file mode 100644
--- /dev/null
+++ b/tests/test-releasenotes-merging.t
@@ -0,0 +1,112 @@
+  $ cat >> $HGRCPATH << EOF
+  > [extensions]
+  > releasenotes=
+  > EOF
+
+  $ hg init simple-repo
+  $ cd simple-repo
+
+A fix directive from commit message is added to release notes
+
+  $ touch fix1
+  $ hg -q commit -A -l - << EOF
+  > commit 1
+  >
+  > .. fix::
+  >
+  >    Fix from commit message.
+  > EOF
+
+  $ cat >> $TESTTMP/single-fix-bullet << EOF
+  > Bug Fixes
+  > =========
+  >
+  > * Fix from release notes.
+  > EOF
+
+  $ hg releasenotes -r . $TESTTMP/single-fix-bullet
+
+  $ cat $TESTTMP/single-fix-bullet
+  Bug Fixes
+  =========
+
+  * Fix from release notes.
+
+  * Fix from commit message.
+
+Processing again will no-op
+TODO this is buggy
+
+  $ hg releasenotes -r . $TESTTMP/single-fix-bullet
+
+  $ cat $TESTTMP/single-fix-bullet
+  Bug Fixes
+  =========
+
+  * Fix from release notes.
+
+    Fix from commit message.
+
+  * Fix from commit message.
+
+  $ cd ..
+
+Sections are unioned
+
+  $ hg init subsections
+  $ cd subsections
+  $ touch fix1
+  $ hg -q commit -A -l - << EOF
+  > Commit 1
+  >
+  > .. feature:: Commit Message Feature
+  >
+  >    This describes a feature from a commit message.
+  > EOF
+
+  $ cat >> $TESTTMP/single-feature-section << EOF
+  > New Features
+  > ============
+  >
+  > Notes Feature
+  > -------------
+  >
+  > This describes a feature from a release notes file.
+  > EOF
+
+  $ hg releasenotes -r . $TESTTMP/single-feature-section
+
+  $ cat $TESTTMP/single-feature-section
+  New Features
+  ============
+
+  Notes Feature
+  -------------
+
+  This describes a feature from a release notes file.
+
+  Commit Message Feature
+  ----------------------
+
+  This describes a feature from a commit message.
+
+Doing it again won't add another section
+
+  $ hg releasenotes -r . $TESTTMP/single-feature-section
+  Commit Message Feature already exists in feature section; ignoring
+
+  $ cat $TESTTMP/single-feature-section
+  New Features
+  ============
+
+  Notes Feature
+  -------------
+
+  This describes a feature from a release notes file.
+
+  Commit Message Feature
+  ----------------------
+
+  This describes a feature from a commit message.
+
+  $ cd ..
diff --git a/tests/test-releasenotes-parsing.t b/tests/test-releasenotes-parsing.t
new file mode 100644
--- /dev/null
+++ b/tests/test-releasenotes-parsing.t
@@ -0,0 +1,170 @@
+  $ cat >> $HGRCPATH << EOF
+  > [extensions]
+  > releasenotes=
+  > EOF
+
+Bullet point with a single item spanning a single line
+
+  $ hg debugparsereleasenotes - << EOF
+  > New Features
+  > ============
+  >
+  > * Bullet point item with a single line
+  > EOF
+  section: feature
+    bullet point:
+      paragraph: Bullet point item with a single line
+
+Bullet point that spans multiple lines.
+
+  $ hg debugparsereleasenotes - << EOF
+  > New Features
+  > ============
+  >
+  > * Bullet point with a paragraph
+  >   that spans multiple lines.
+  > EOF
+  section: feature
+    bullet point:
+      paragraph: Bullet point with a paragraph that spans multiple lines.
+
+  $ hg debugparsereleasenotes - << EOF
+  > New Features
+  > ============
+  >
+  > * Bullet point with a paragraph
+  >   that spans multiple lines.
+  >
+  >   And has an empty line between lines too.
+  >   With a line cuddling that.
+  > EOF
+  section: feature
+    bullet point:
+      paragraph: Bullet point with a paragraph that spans multiple lines.
+      paragraph: And has an empty line between lines too. With a line cuddling that.
+
+
+Multiple bullet points. With some entries being multiple lines.
+
+  $ hg debugparsereleasenotes - << EOF
+  > New Features
+  > ============
+  >
+  > * First bullet point. It has a single line.
+  >
+  > * Second bullet point.
+  >   It consists of multiple lines.
+  >
+  > * Third bullet point. It has a single line.
+  > EOF
+  section: feature
+    bullet point:
+      paragraph: First bullet point. It has a single line.
+      paragraph: Second bullet point. It consists of multiple lines.
+      paragraph: Third bullet point. It has a single line.
+
+Bullet point without newline between items
+
+  $ hg debugparsereleasenotes - << EOF
+  > New Features
+  > ============
+  >
+  > * First bullet point
+  > * Second bullet point
+  >   And it has multiple lines
+  > * Third bullet point
+  > * Fourth bullet point
+  > EOF
+  section: feature
+    bullet point:
+      paragraph: First bullet point
+      paragraph: Second bullet point And it has multiple lines
+      paragraph: Third bullet point
+      paragraph: Fourth bullet point
+
+Sub-section contents are read
+
+  $ hg debugparsereleasenotes - << EOF
+  > New Features
+  > ============
+  >
+  > First Feature
+  > -------------
+  >
+  > This is the first new feature that was implemented.
+  >
+  > And a second paragraph about it.
+  >
+  > Second Feature
+  > --------------
+  >
+  > This is the second new feature that was implemented.
+  >
+  > Paragraph two.
+  >
+  > Paragraph three.
+  > EOF
+  section: feature
+    subsection: First Feature
+      paragraph: This is the first new feature that was implemented.
+      paragraph: And a second paragraph about it.
+    subsection: Second Feature
+      paragraph: This is the second new feature that was implemented.
+      paragraph: Paragraph two.
+      paragraph: Paragraph three.
+
+Multiple sections are read
+
+  $ hg debugparsereleasenotes - << EOF
+  > New Features
+  > ============
+  >
+  > * Feature 1
+  > * Feature 2
+  >
+  > Bug Fixes
+  > =========
+  >
+  > * Fix 1
+  > * Fix 2
+  > EOF
+  section: feature
+    bullet point:
+      paragraph: Feature 1
+      paragraph: Feature 2
+  section: fix
+    bullet point:
+      paragraph: Fix 1
+      paragraph: Fix 2
+
+Mixed sub-sections and bullet list
+
+  $ hg debugparsereleasenotes - << EOF
+  > New Features
+  > ============
+  >
+  > Feature 1
+  > ---------
+  >
+  > Some words about the first feature.
+  >
+  > Feature 2
+  > ---------
+  >
+  > Some words about the second feature.
+  > That span multiple lines.
+  >
+  > Other Changes
+  > -------------
+  >
+  > * Bullet item 1
+  > * Bullet item 2
+  > EOF
+  section: feature
+    subsection: Feature 1
+      paragraph: Some words about the first feature.
+    subsection: Feature 2
+      paragraph: Some words about the second feature. That span multiple lines.
+    bullet point:
+      paragraph: Bullet item 1
+      paragraph: Bullet item 2


_______________________________________________
Mercurial-devel mailing list
[hidden email]
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Loading...