Bugslist
Bugslist is the ProgClub TODO list management software. That's the software that generates a TODO list from the comments in your source code. If you use this script you might be interested in ProgClub Timestamp so you can copy/paste the comment label syntax, or configure KDE so you don't have to. For other projects see projects.
// 2023-05-01 jj5 - TODO: see this (scroll down), we should add 'KLUDGE' as an alias for 'HACK'.
Status
We use semantic versioning. Latest production version: 0.2. Latest development version: 0.2.
See the TODO list for work that still needs to be done.
Motivation
Why this software? This is so that we can put TODO notes in our source code where they're most relevant and convenient but also publish a TODO list on the wiki and in our documentation.
Administration
Contributors
Members who have contributed to this project. Newest on top.
All contributors have agreed to the terms of the Contributor License Agreement. This excludes any upstream contributors who tend to have different administrative frameworks.
Copyright
Copyright © 2020, Contributors.
License
Licensed under the GPL.
Components
Libraries, tools, services or media from third parties used under license:
Resources
Downloads
There are presently no tarballs. Running an `svn checkout` against the ProgClub Subversion repository (pcrepo) is the recommended way to install this software. See the Installation Guide for details.
Source code
The source code can be browsed online:
The most interesting code is here:
The latest stable (read-only) released version of the code is available from Subversion here:
Or if you want the latest version for development purposes:
Note that our software development is done on the version branch, not on trunk. We use trunk to track the latest stable release.
Specifications
Functional specification
The functional specification describes what the project does.
See help for information about how the script is supposed to work.
Technical specification
The technical specification describes how the project works.
The script is a PHP CLI script (tested on PHP v7.2). It shells out to Subversion for some of its functionality.
Notes
See versioning for information on our semantic version numbers.
Notes for implementers
If you are interested in incorporating this software into your project, here's what you need to know.
Installation
This software has been developed and tested on Ubuntu 18.04 LTS GNU/Linux. It will probably work in other Unix environments and it will definitely be broken on Windows. (If you'd like to make it work on Windows we will accept your patch!)
After you install dependencies you need to decide if you want to install for just your user or for an entire system.
After you've installed bugslist swing by the ProgClub Timestamp utility for some documentation concerning KDE shortcut key config to help you enter bugslist compatible comments in your code.
Dependency installation
In order to run bugslist you might also need to install some dependencies, if so do that first:
$ sudo apt install php-cli subversion
System installation
After you install dependencies you can install bugslist for an entire system with these two commands:
$sudo svn checkout https://www.progclub.org/svn/pcrepo/bugslist/trunk /usr/local/lib/bugslist
$sudo ln -s /usr/local/lib/bugslist/bin/bugslist.php /usr/local/bin/bugslist
System upgrade
If you've done a system installation you can do a system upgrade like this:
$ sudo svn up /usr/local/lib/bugslist
User installation
First make sure you've installed the dependencies.
The recommended way to install this software is by checking out the 'trunk' version from source control.
In order to run bugslist you should create a bugslist
symlink somewhere in your $PATH and point it to bin/bugslist.php
.
So a full installation might look something like this:
$cd ~/software
$svn checkout https://www.progclub.org/svn/pcrepo/bugslist/trunk bugslist
$cd ~/bin
$ln -s ~/software/bugslist/bin/bugslist.php bugslist
User upgrade
Assuming you followed the user installation instructions you can do a user upgrade something like this:
$ svn up ~/software/bugslist
Configuration
After you install bugslist you might like to configure one or more projects. Create a config file called etc/bugslist.php
under the directory you will run bugslist in. See etc/bugslist-example.php
for an example.
Notes for developers
If you're looking to set up a development environment for this project here's what you need to know:
Get the latest code from the version branch:
$ svn checkout https://www.progclub.org/svn/pcrepo/bugslist/branches/0.2 bugslist-0.2
You can then run the unit tests like this:
$ cd bugslist-0.2 && bin/test/test.sh
Note that you may need to install some dependencies, such as:
# apt install php-cli subversion
If you have a bugfix or improvement please send your patch to jj5@progclub.org along with a statement that you are willing to be listed in the contributors section of the documentation and willing to license your contribution under the license.
Notes for administrators
To release a version of this project use svnman.
First, run maintenance and commit any changes:
$ svnman commit
Then run the release:
$ bin/dev/release.sh
You will be prompted to update the project documentation.
Miscellanea
Subversion
The bugslist software makes use of the Subversion command-line interface. If Subversion is not installed then bugslist may not work!
ViewVC
The ViewVC software can be used for browsing a Subversion repository via the web. We support ViewVC integration (via the config file).
BASH
The bash shell is our Unix shell of choice. You don't need Bash to run bugslist but it is used by our build scripts.
PHP
The bugslist software is written mostly in the PHP programming language which we^H^HI love! :)
Other bits and pieces are done with Subversion and BASH.
Ubuntu
This software was developed and tested on Ubuntu 18.04 LTS.
$PATH
When you install bugslist you should make sure a symlink to bin/bugslist.php
is in your $PATH environment variable.
Generated documentation
Help
Usage: php path/to/bugslist.php [OPTION]... [PATH]... This is a source code processing and reporting tool to help with software development project management. It processes files in the specified directory and matches special comment syntax to get notes and work items which it then reports on. Output reports are in MediaWiki wiki text syntax designed to be included in your project wiki.
Comment label syntax
The heart of bugslist is the comment label syntax you use in your source code. It's basically: DATE USER DASH TYPE[PRIORITY][:!] TEXT For example: 2020-03-04 jj5 - HACK[HIGH]: expedient, you say? The date is in ISO format, the user must be a valid UNIX user, and the dash is just to try and limit false positives. The TYPE indicates what sort of item this is; it can be one of the types listed below. Your type can be suffixed with an optional priority within square brackets. After the optional priority your type must be suffixed with punctuation, either ':' or '!'. The TEXT follows until the end of the line and on to subsequent lines too.
Item types
The following item types are supported in comment labels:
Type................: Description..............................: Priority..: Red-flags..........: TEMP..............: a note for code that should be removed...: HIGH Stuff to do........: BUG...............: something that needs to be fixed.........: HIGH TODO..............: something that needs to be done..........: MEDIUM FIXME............: a HIGH priority alias for TODO...........: HIGH NICE.............: a LOW priority alias for TODO............: LOW TEST..............: something that needs to be tested........: MEDIUM HACK..............: something that needs to be improved......: LOW THINK.............: something to consider or think about.....: LOW CONSIDER.........: an alias for THINK.......................: LOW Stuff that's done..: FIXED.............: a BUG that has been fixed................: HIGH DONE..............: something that has been done.............: MEDIUM COMMIT............: a VCS commit log.........................: LOW Notes..............: WARNING...........: something to be careful of...............: HIGH REFERENCE.........: a reference, e.g. a URL..................: MEDIUM REF..............: an alias for REFERENCE...................: MEDIUM SEE..............: an alias for REFERENCE...................: MEDIUM NOTE..............: notes, or items without a label..........: LOW DEBUG.............: an item for debugging purposes...........: DEBUG
Item priority
You can use a priority number, priority letter, or priority name to indicate a priority: #: Abbr: Name: Default: 0: X: NEXT: N/A 1: !, C: CRITICAL: N/A 2: H: HIGH: TEMP, BUG, FIXME, FIXED, WARNING 3: M: MEDIUM: TODO, TEST, DONE, REFERENCE[REF|SEE] 4: L: LOW: NICE, HACK, THINK[CONSIDER], COMMIT, NOTE 9: D: DEBUG: DEBUG
Comment syntax
Comment syntax for various programming languges is supported: Plain text...: comment labels begin at the first character on the line C++, etc //..: comment labels begin after '// ' (the space is important) HTML/XML.....: comment labels are within: <!-- --> Perl #.......: comment labels begin after '# ' (the space is important) C/CSS........: comment labels are within: /* */ Bind, INI....: comment labels begin after '; ' (the space is important)
Program options
File-system options
-i, --include=EXTENSIONS comma-separated list of included file extensions -e, --exclude=EXTENSIONS comma-separated list of excluded file extensions Note: excludes override includes. This is useful when you want to exclude missing extensions () which are included by default. -s, --skip=REGEXES comma-separated list of regexes for paths to skip -k, --keep=REGEXES comma-separated list of regexes for paths to keep ** BE VERY CAREFUL ** Case sensitive expressions. Use '^' to match start of path, '$' to match end of path, etc. Note: keeps override skips
Filtering options
-u, --user=USER include items for USER -d, --date=DATE include items for DATE --today include items for today only (default if not debugging) --yesterday include items for yesterday only -t, --type=TYPE include items by type, will affect report order. Duplicates are ignored. -y, --priority=PRIORITY include items with particular PRIORITY. --all report on all users, dates, types, and priorities. --none don't report matches (stats only).
Reporting options
-a, --aggregate=FIELD report aggregation. Duplicates are ignored. A heading is printed for each aggregated field. -c, --collate=FIELD report sorting. Duplicates are ignored. Default sorting uses binary comparision rather than locale sensitive comparison which is probably what you want. For --aggregate and --collate uppercase FIELD sorts descending. e.g. 'dUp' selects by date, user desc, and path -h, --heading=FORMAT heading format options. Duplicates are ignored. -o, --output=FIELD|FORMAT report output options. Duplicates are ignored. -l, --level level of heading to start at (default 2) --na output 'N/A' when no report data +N, --natural use locale sensitive sorting instead of binary sorting +B, --binary use binary sorting (This is the default. Redundant? :) +Q, --quiet disable warnings, stats, and progress on stderr +S, --show-stats enable statistics report on stderr -S, --hide-stats disable statistics report on stderr (default) +C, --show-loc enable lines of code report on stderr -C, --hide-loc disable lines of code report on stderr (default) +T, --show-ext enable file type report on stderr -T, --hide-ext disable file type report on stderr (default) +P, --show-progress enable progress report on stderr -P, --hide-progress disable progress report on stderr (default) +O, --show-options report on user options on stderr -O, --hide-options disable user options report (default) +W, --show-warnings report warnings on stderr (default) -W, --hide-warnings disable warnings
Formatting options
--text output as plain-text (default) // 2017-06-06 jj5 - TODO: finish support for text output --wiki output as MediaWiki wiki-text // 2017-06-06 jj5 - TODO: finish support for Wiki output --html output as HTML // 2017-06-06 jj5 - TODO: add support for HTML output
--list output as list (default) // 2017-06-06 jj5 - TODO: add support for HTML lists --table output as table (instead of list) // 2017-06-06 jj5 - TODO: add support for tables
Count options
--items print the total number of items and exit --tasks print the total number of tasks and exit --fixes print the total number of fixes and exit --features print the total number of features and exit --commits print the total number of commits and exit --notes print the total number of notes and exit
Other options
--help display this help text and exit --web-help open documentation in Firefox --version display version information and exit +V, --vcs parse version control (Subversion) logs +D, --debug some handy defaults, when debugging or not +X, --exit exit without reporting (useful with --show-stats etc) -w, --slow=MICROTIME process items slowly (delay MICROTIME per file) -m, --publish=PUBNAME publish output to a MediaWiki wiki (requires config). Think 'm' for 'subMit'... --unknown display list of unknown file extensions then exit
Format specifications
USER
USER is comma-separated list of usernames. Use '*' for all users; '?' for current user.
DATE
DATE is ISO formatted date prefix, e.g.: '2016' for all items in 2016, '2016-01' for items in Jan 2016, '2016-,01,02' for items in Jan/Feb 2016, '2016-01-,01,02' for items on 1st/2nd Jan 2106, '2016-01-02,2016-01-04' for 2nd/4th Jan 2016. Use '*' for all dates.
TYPE
TYPE is a string of abbreviations or comma-separated list of item types. Supported types are: Red-flags: m: TEMP.......: a note for code that should be removed Stuff to do: b: BUG........: something that needs to be fixed t: TODO.......: something that needs to be done e: TEST.......: something that needs to be tested h: HACK.......: something that needs to be improved c: THINK......: something to consider or think about Stuff that's done: f: FIXED......: a BUG that has been fixed d: DONE.......: something that has been done v: COMMIT.....: a VCS commit log Notes: w: WARNING....: something to be careful of r: REFERENCE..: a reference, e.g. a URL s: REFERENCE..: an alias for 'r' n: NOTE.......: notes, or items without a label g: DEBUG......: an item for debugging purposes e.g. 'TODO, NOTE' or 'bht' for 'BUG, HACK, TODO' Use '*' for all types.
PRIORITY
PRIORITY is a comma-separeted list of priorities or a range of priorities indicated with a dash '-' between a low and a high priority. A priority can be indicated by a priority number, a priority letter or abbreviation, or a priority name. Use '*' for all priorities. Supported values are: #: Abbr: Name...: 0: X: NEXT 1: !, C: CRITICAL 2: H: HIGH 3: M: MEDIUM 4: L: LOW 9: D: DEBUG
FIELD
FIELD is a string of abbreviations or comma-separated list of fields. Supported fields are: u: user: select username d: date: select date g: long-date: select date in long format t: type: select item type y: priority: select item priority e: filemtime: select file modification timestamp b: both: select path and file p: path: select path f: file: select file, do not prefix path. This may not be what you want, may merge files from different directories. l: line: select line number m: name: select file name k: link: select link to file (for specific revision) s: reference: select reference (or text if not REFERENCE) i: initials: select Wiki initials initials (requires config) n: new-lines: select number of new lines o: old-lines: select number of old lines c: line-diff: select lines difference r: revision: select VCS revision number w: code: select work code x: text: select item text
FORMAT
FORMAT is a string of abbreviations or comma-separated list of format options. Supported format options are: U: plain-user: don't format user D: plain-date: don't format date T: plain-type: don't format type Y: plain-priority: don't format priority E: plain-filemtime: don't format filemtime B: plain-both: don't format path/file P: plain-path: don't format path F: plain-file: don't format file L: plain-line: don't format line M: plain-name: don't format name K: plain-link: don't format link S: plain-reference: don't format reference I: plain-initials: don't format initials N: plain-new-lines: don't format new lines O: plain-old-lines: don't format old lines C: plain-line-diff: don't format line diff R: plain-revision: don't format VCS revision number W: plain-code: don't format work code X: plain-text: don't format item text e.g. 'UT' disables formatting for user and type values.
PUBNAME
PUBNAME is string that represents the publishing options to use. These options must be configured in the config file.
// 2017-06-06 jj5 - TODO: add support for MediaWiki publishing.
MediaWiki integration
For proper wikitable support edit MediaWiki:Common.css and add:
.right td { text-align: right; } .right td:first-child { text-align: left; }
Feedback
Report bugs to: jj5@progclub.org
Links
Generated with bugslist.
- bugslist.php: Bugslist Installation Guide
- bugslist.php: bugslist home page
- bugslist.php: PHP set_exception_handler()
- bugslist.php: PHP set_error_handler()
- bugslist.php: PHP preg_match()
- bugslist.php: PHP error constants
TODO list
Generated with bugslist.
HIGH priority
- BUG: bugslist.php: plain and not-html-encoded need to be separated. They are conflated at the moment. Perhaps a 'text' reporting option..?
- TODO: bugslist.php: need to check we're not processing svn property changes...
- TODO: bugslist.php: it would be better to use strrpos to find the last colon...
MEDIUM priority
- TODO: bugslist.php: check if 'ods' is plain text / XML format...
- TODO: bugslist.php: document --max-date
- TODO: bugslist.php: unit tests for --max-date
- TODO: bugslist.php: add --min-date
- TODO: bugslist.php: implement revision filter
- TODO: bugslist.php: finish support for text output
- TODO: bugslist.php: finish support for Wiki output
- TODO: bugslist.php: add support for HTML output
- TODO: bugslist.php: add support for HTML lists
- TODO: bugslist.php: add support for tables
- TODO: bugslist.php: add support for MediaWiki publishing.
- TODO: bugslist.php: call is_int( $line ) instead. Confirm $line is an integer.
- TODO: bugslist.php: HTML table reporting
- TODO: bugslist.php: HTML list reporting
- TODO: bugslist.php: wiki table reporting
- TODO: bugslist.php: check for newlines in value and remove them
- TODO: bugslist.php: make priority filter and default priority a config file option.
- TODO: bugslist.php: make $file_extension_include, etc. configurable in config file.
- TODO: bugslist.php: make config override a command-line option. e.g.
- --config=user,jj5,initials=JE;user,jj5,wiki-user=Sixsigma
- TODO: bugslist.php: ignore extra punctuation after item type/priority
- TODO: bugslist.php: VCS support should probably include git in addition to svn.
- TODO: bugslist.php: test duplicate command-line options
- TODO: bugslist.php: add better support for multiline svn commit messages
- TODO: bugslist.php: update doco about skip/keep now using regular expressions.
- TODO: bugslist.php: change path to prefix '/'
- TODO: bugslist.php: use <nowiki></nowiki> for ..? Or henc() item text? Or surround item text in <nowiki>?
- TODO: bugslist.php: check for 'basic' regex match and warn if we don't match a proper item. I.e. we might match '//DATE USER -' (without space after '//' which will be skipped. We should warn in that case.
- TODO: bugslist.php: add support for item age (in days)
- TODO: bugslist.php: add support for svn:externals...
- TODO: bugslist.php: add a dates list report (as an index into e.g. ((MyProject/2016-12-22|2016-12-22)), etc.)
- TODO: bugslist.php: add support for --max-date and --min-date
- TODO: bugslist.php: support for START/END validation
- TODO: bugslist.php: prefix item text with 'wiki: ' to flag item as being encoded in wiki text...
- TODO: bugslist.php: add a LOC count feature, by file type, direcoty, etc.
- TODO: bugslist.php: detected VCS could be svn *and* git... use an array.
- TODO: bugslist.php: match START/END notes, warn mismatch, and ignore them.
- TODO: bugslist.php: add support for '[[#NEW|NEW]]' notes for patches
- TODO: bugslist.php: add support for '[[#OLD|OLD]]' notes for removals
- TODO: bugslist.php: match {NEW,OLD}/END notes
- TODO: bugslist.php: --text, --html, --wiki output modes
- TODO: bugslist.php: --list, --table formatting options
- TODO: bugslist.php: add support for a '[[#UPGRADE|UPGRADE]]' type which are things that need to be reviewed on upgrade.
- TODO: bugslist.php: get rid of -p and -f options and just process all files/directories nominated on the command line.
- TODO: bugslist.php: need to improve the wiki documentation, at the moment there are a lot of broken internal links to missing content. We probably need to add a 'gen-doc' feature to the bugslist command.
- TODO: bugslist.php: we should make sure that bugslist can run even if the 'svn' executable is missing.
- TODO: run.sh: plain-* output options
- TODO: run.sh: test --na and -o=X and -o=g
- TODO: run.sh: review this test and re-enable it if it's useful.
LOW priority
- TODO: bugslist.php: not sure if a little sleep for a few milliseconds is helpful or not..? Should do some benchmarking...
- TODO: bugslist.php: linting for invalid comment labels, e.g. HAKK, missing/invalid punctuation, etc.
- TODO: bugslist.php: MediaWiki integration would be nice. I.e. automatically insert reports into the MediaWiki database. Might be smarter to go in through the front-end via HTTP (need HTTP auth and a valid cookie for that... doable.)
- TODO: bugslist.php: add revision range filters, e.g. -r=50:100
- TODO: bugslist.php: renumber user exit levels to list file-system stuff before filters stuff.
- HACK: bugslist.php: we just bail if previous revision is zero.
- HACK: bugslist.php: We haven't verfied the syntax of the URL, and it might require HTML encoding?
- HACK: bugslist.php: We haven't verfied the syntax of the URL, and it might require HTML encoding?
- HACK: bugslist.php: OK, so if we ask to report references on items which are not references we will just report the item text:
- THINK: bugslist.php: do we want either of these..?
- THINK: bugslist.php: make DEBUG 'x' or 'z'..?
- THINK: bugslist.php: maybe default CONSIDER to MEDIUM..?
- THINK: bugslist.php: consider support for 'Z' => '[[#DEBUG|DEBUG]]'..?
- THINK: bugslist.php: consider if 'phar' should be included...
- THINK: bugslist.php: consider if 'properties' should be excluded...
- THINK: bugslist.php: consider if 'xcf' should be excluded...
- THINK: bugslist.php: does $proc_descriptor need to be a global..?
- THINK: bugslist.php: would it be better to scan for nested TODO items when $type is finished {FIXED,DONE,COMMIT} and associate the details with the finished item..?
- THINK: bugslist.php: make null headings a command-line/config option? At the moment null headings are skipped, not reported.
- THINK: bugslist.php: is this what we want..?
- THINK: bugslist.php: alias '[[#TASK|TASK]]' to '[[#TODO|TODO]]'..?
- THINK: bugslist.php: add support for FEATURE/IMPROVEMENT items..?
- THINK: bugslist.php: add support for ';' list delimiters in addition to ','..?
- THINK: bugslist.php: add support for multiple reports delimited with ';'..?
- THINK: bugslist.php: add MAINT type w/ --maint option to report on maintenance gotchas? (i.e. stuff you need to know about for doing a release etc.)
- THINK: bugslist.php: for svn items is the line number 'svn' or is it the first changed line in the associated file? (2016-12-15 jj5: I think the line number is the revision number, which is handy if we sort by line.)
- THINK: bugslist.php: add '[[#SOON|SOON]]' item type?
- THINK: bugslist.php: add BUGREF type, for references to include in BUG reports..? At the moment we've just extended BUG to include an optional URL...
- THINK: bugslist.php: remove START/END notes from notes..?
- THINK: bugslist.php: do we wanna skip symlinks when processing..?