Plugin tutorial ready

Still missing: reference.

1 files changed, 299 insertions, 0 deletions

diff --git a/doc/plugins.md b/doc/plugins.md
new file mode 100644
index 0000000..df454a2
--- /dev/null
+++ b/doc/plugins.md
@@ -0,0 +1,299 @@
+#### TODO ####
+
+- write a rebar3 template for plugin writing, make it easier on our poor souls
+- rework the tutorial to use the rebar3 template for plugins
+
+# Plugins #
+
+Rebar3's system is based on the concept of
+*[providers](https://github.com/tsloughter/providers)*. A provider has three
+callbacks:
+
+- `init(State) -> {ok, NewState}`, which helps set up the state required, state dependencies, etc.
+- `do(State) -> {ok, NewState} | {error, String}`, which does the actual work.
+- `format_error(Error, State) -> {String, NewState}`, which allows to print errors
+  when they happen, and to filter out sensitive elements from the state.
+
+A provider should also be an OTP Library application, which can be fetched as
+any other Erlang dependency, except for Rebar3 rather than your own system or
+application.
+
+This document contains the following elements:
+
+- [Using a Plugin](#using-a-plugin)
+- [Reference](#reference)
+  - [Provider Interface](#provider-interface)
+  - [List of Possible Dependencies](#list-of-possible-dependencies)
+  - [Rebar State Manipulation](#rebar-state-manipulation)
+- [Tutorial](#tutorial)
+
+## Using a Plugin ##
+
+## Reference ##
+
+### Provider Interface ###
+
+### List of Possible Dependencies ###
+
+### Rebar State Manipulation ###
+
+## Tutorial ##
+
+### First version ###
+In this tutorial, we'll show how to start from scratch, and get a basic plugin
+written. The plugin will be quite simple: it will look for instances of 'TODO:'
+lines in comments and report them as warnings. The final code for the plugin
+can be found on [bitbucket](https://bitbucket.org/ferd/rebar3-todo-plugin).
+
+The first step is to create a new OTP Application that will contain the plugin:
+
+    → git init
+    Initialized empty Git repository in /Users/ferd/code/self/rebar3-todo-plugin/.git/
+    → mkdir src
+    → touch src/provider_todo.erl src/provider_todo.app.src
+
+Let's edit the app file to make sure the description is fine:
+
+```erlang
+{application, provider_todo, [
+  {description, "example rebar3 plubin"},
+  {vsn, "0.1.0"},
+  {registered, []},
+  {applications, [kernel, stdlib]},
+  {env, []}
+]}.
+```
+
+Open up the `provider_todo.erl` file and make sure you have the following
+skeleton in place:
+
+```erlang
+-module(provider_todo).
+-behaviour(provider).
+
+-export([init/1, do/1, format_error/2]).
+
+-include_lib("rebar3/include/rebar.hrl").
+
+-define(PROVIDER, todo).
+-define(DEPS, [app_discovery]).
+
+%% ===================================================================
+%% Public API
+%% ===================================================================
+-spec init(rebar_state:t()) -> {ok, rebar_state:t()}.
+init(State) ->
+    Provider = providers:create([
+            {name, ?PROVIDER},          % The 'user friendly' name of the task
+            {module, ?MODULE},          % The module implementation of the task
+            {bare, true},               % The task can be run by the user, always true
+            {deps, ?DEPS},              % The list of dependencies
+            {example, "rebar $PLUGIN"}, % How to use the plugin
+            {opts, []}                  % list of options understood by the plugin
+            {short_desc, ""},
+            {desc, ""}
+    ]),
+    {ok, rebar_state:add_provider(State, Provider)}.
+
+
+-spec do(rebar_state:t()) -> {ok, rebar_state:t()} | {error, string()}.
+do(State) ->
+    {ok, State}.
+
+-spec format_error(any(), rebar_state:t()) ->  {iolist(), rebar_state:t()}.
+format_error(Reason, State) ->
+    {io_lib:format("~p", [Reason]), State}.
+```
+
+This shows all the basic content needed. Note that we leave the `DEPS` macro to
+the value `app_discovery`, used to mean that the plugin should at least find
+the project's source code (excluding dependencies).
+
+In this case, we need to change very little in `init/1`. Here's the new
+provider description:
+
+```erlang
+        Provider = providers:create([
+            {name, ?PROVIDER},       % The 'user friendly' name of the task
+            {module, ?MODULE},       % The module implementation of the task
+            {bare, true},            % The task can be run by the user, always true
+            {deps, ?DEPS},           % The list of dependencies
+            {example, "rebar todo"}, % How to use the plugin
+            {opts, []},              % list of options understood by the plugin
+            {short_desc, "Reports TODOs in source code"},
+            {desc, "Scans top-level application source and find "
+                   "instances of TODO: in commented out content "
+                   "to report it to the user."}
+    ]),
+```
+
+Instead, most of the work will need to be done directly in `do/1`. We'll use the
+`rebar_state` module to fetch all the applications we need. This can be done by
+calling the `project_apps/1` function, which returns the list of the project's
+top-level applications.
+
+```erlang
+do(State) ->
+    lists:foreach(fun check_todo_app/1, rebar_state:project_apps(State)),
+    {ok, State}.
+```
+
+This, on a high level, means that we'll check each top-level app one at a time
+(there may often be more than one top-level application when working with
+releases)
+
+The rest is filler code specific to the plugin, in charge of reading each
+app path, go read code in there, and find instances of 'TODO:' in comments
+in the code:
+
+```erlang
+check_todo_app(App) ->
+    Path = filename:join(rebar_app_info:dir(App),"src"),
+    Mods = find_source_files(Path),
+    case lists:foldl(fun check_todo_mod/2, [], Mods) of
+        [] -> ok;
+        Instances -> display_todos(rebar_app_info:name(App), Instances)
+    end.
+
+find_source_files(Path) ->
+    [filename:join(Path, Mod) || Mod <- filelib:wildcard("*.erl", Path)].
+
+check_todo_mod(ModPath, Matches) ->
+    {ok, Bin} = file:read_file(ModPath),
+    case find_todo_lines(Bin) of
+        [] -> Matches;
+        Lines -> [{ModPath, Lines} | Matches]
+    end.
+
+find_todo_lines(File) ->
+    case re:run(File, "%+.*(TODO:.*)", [{capture, all_but_first, binary}, global, caseless]) of
+        {match, DeepBins} -> lists:flatten(DeepBins);
+        nomatch -> []
+    end.
+
+display_todos(_, []) -> ok;
+display_todos(App, FileMatches) ->
+    io:format("Application ~s~n",[App]),
+    [begin
+      io:format("\t~s~n",[Mod]),
+      [io:format("\t  ~s~n",[TODO]) || TODO <- TODOs]
+     end || {Mod, TODOs} <- FileMatches],
+    ok.
+```
+
+Just using `io:format/2` to output is going to be fine.
+
+To test the plugin, push it to a source repository somewhere. Pick one of your
+projects, and add something  to the rebar.config:
+
+```erlang
+{plugins, [
+  {provider_todo, ".*", {git, "git@bitbucket.org:ferd/rebar3-todo-plugin.git", {branch, "master"}}}
+]}.
+```
+
+Then you can just call it directly:
+
+```
+→ rebar3 todo
+===> Fetching provider_todo
+Cloning into '.tmp_dir539136867963'...
+===> Compiling provider_todo
+Application merklet
+    /Users/ferd/code/self/merklet/src/merklet.erl
+      todo: consider endianness for absolute portability
+```
+
+Rebar3 will download and install the plugin, and figure out when to run it.
+Once compiled, it can be run at any time again.
+
+### Optionally Search Deps ###
+
+Let's extend things a bit. Maybe from time to time (when cutting a release),
+we'd like to make sure none of our dependencies contain 'TODO:'s either.
+
+To do this, we'll need to go parse command line arguments a bit, and
+change our execution model. The `?DEPS` macro will now need to specify that
+the `todo` provider can only run *after* dependencies have been installed:
+
+```erlang
+-define(DEPS, [install_deps]).
+```
+
+We can add the option to the list we use to configure the provider in `init/1`:
+
+```erlang
+{opts, [                 % list of options understood by the plugin
+    {deps, $d, "deps", undefined, "also run against dependencies"}
+]},
+```
+
+Meaning that deps can be flagged in by using the option `-d` (or `--deps`), and
+if it's not defined, well, we get the default value `undefined`. The last
+element of the 4-tuple is documentation for the option.
+
+And then we can implement the switch to figure out what to search:
+
+```erlang
+do(State) ->
+    Apps = case discovery_type(State) of
+        project -> rebar_state:project_apps(State);
+        deps -> rebar_state:project_apps(State) ++ rebar_state:src_deps(State)
+    end,
+    lists:foreach(fun check_todo_app/1, Apps),
+    {ok, State}.
+
+[...]
+
+discovery_type(State) ->
+    {Args, _} = rebar_state:command_parsed_args(State),
+    case proplists:get_value(deps, Args) of
+        undefined -> project;
+        _ -> deps
+    end.
+```
+
+The `deps` option is found using `rebar_state:command_parsed_args(State)`,
+which will return a proplist of terms on the command-line after 'todo',
+and will take care of validating whether the flags are accepted or not. The
+rest can remain the same.
+
+Push the new code for the plugin, and try it again on a project with
+dependencies:
+
+```
+→ rebar3 todo --deps
+===> Fetching provider_todo
+Cloning into '.tmp_dir846673888664'...
+===> Compiling provider_todo
+===> Fetching bootstrap
+Cloning into '.tmp_dir57833696240'...
+===> Fetching file_monitor
+Cloning into '.tmp_dir403349997533'...
+===> Fetching recon
+Cloning into '.tmp_dir390854228780'...
+[...]
+Application dirmon
+    /Users/ferd/code/self/figsync/apps/dirmon/src/dirmon_tracker.erl
+      TODO: Peeranha should expose the UUID from a node.
+Application meck
+    /Users/ferd/code/self/figsync/_deps/meck/src/meck_proc.erl
+      TODO: What to do here?
+      TODO: What to do here?
+```
+
+Rebar3 will now go pick dependencies before running the plugin on there.
+
+you can also see that the help will be completed for you:
+
+```
+→ rebar3 help todo
+Scans top-level application source and find instances of TODO: in commented out content to report it to the user.
+
+Usage: rebar todo [-d]
+
+  -d, --deps  also run against dependencies
+```
+
+That's it, the todo plugin is now complete! It's ready to ship and be
+included in other repositories.