DevHeads.net

git workflow draft

hi all....

so after the meeting on Sunday, here is where we are in terms of a draft
workflow. more complete meeting minutes can be seen here:

<a href="http://titanpad.com/SnJwFW2iXL" title="http://titanpad.com/SnJwFW2iXL">http://titanpad.com/SnJwFW2iXL</a>

the goal of the meeting was to come up with a draft of a mutually agreeable
git workflow for kdelibs and kde-runtime. the hope is that this can become a
template for the rest of the SC modules (kde-workspace will follow the kdelibs
workflow, for instance), as well as as many other KDE projects as possible.
this is to help ensure a general continuity to how we work across KDE.

Topic Branches
=============

All new development should happen in a branch. Git makes branches very cheap
and they can be local or remote. There are few if any really good reasons not
to use branches, so development directly in master should be generally
discouraged.

Topic / feature branches should be public and pushed to the main repository
where they are easy for others to find and collaborate on. They should start
as a branch off of master. We do not want git to become, even unintentionally,
a road to segregated, private development at the expense of our collaboration
as a community. With public branches in a shared repository, even a git pull
will inform of new development that is happening. Git then becomes an
important piece of our development communication with each other: a new branch
means new activity.

Only features / topics that are intended from the state to be merged with
master should end up in the main repository, however. More experimental and/or
long term efforts (an example presented was the kconfig refactor leading up to
4.0) should start in a personal clone, and when/if the work crosses the border
into "this is realistically going to be merged with master" it can be moved
into a branch in the main repository.

After merge with master, topic branches should be deleted from the main
repository.

Branch Naming: if a branch is specific to a subproject, e.g. solid, specify it
in the branch name such as "solid/udevbackend". This will make it easier to
use git branch listings (along with grep, etc) to pick out branches of
interest based on the project in question. If there is not a sepcific
subproject that it belongs to, give it a good descriptive name such as
"pluggable-kconfig".

No branches should be prefixed with "KDE"; we consider that a reserved name.
Nor topic should branches be numeric in nature (such as a version number) as
those are reserved for actual release branches. (Sys admin at the meeting
indicated that it is likely they will eventually put a push hook in place that
prevents such "incorrectly" named branches.)

Branch names should not include the committer name as we have git log and it
will help discourage "ego coding" where one person has overt "ownership" of
the branch. This is still a team effort after all :)

"Dead" branches: on every maor every release, a check will be done on branches
in the main repository. Any branches that have not seen any work done on them
in the last <TBD: 12? 18?> months will be deleted. Deleted branches are
backed up automatically on git.kde.org, so work will be retrievable. This will
keep our branch count sane over the years.

Open questions / topics for further discovery:

* documenting best practices for keeping a topic branch in sync with
master, keeping in mind that later a merge from the topic branch to master
needs to be done and the git history sould be kept as clean as possible

* process for announcing that a branch is ready to be merged so as to
gather feedback (particularly important for the shared "crown jewels" in
kdelibs); this is something like a "kdereview 2.0" though on the level of
feature branches.

* topic branches and our release cycle: is it acceptable to create a new
topic branch even when we are in feature freeze?

* more advanced workflows involving, e.g., an integration branch is not
something we seem equipped for right now as it brings overhead and additional
coordination with it. Perhaps in the future though ....

* best practice "recipes" for merging, e.g. using "git merge --log"

Bugfixes
=======
The big question here was forward-porting vs. backporting. Backporting has the
advantage that it is what we do currently and people run and test master more.
Forward-porting ensures that all bug fixes in the stable release branch end up
in the master branch and it is easier to test against a branch with fewer
changes (e.g. new features). It will take discipline either way: two bug fixes
we found in 4.6 that weren't in master.

Merge is prefered over cherry-pick: it cleans up the git history a bit; each
commit is only there once instead of the cloned commits created by cherry-
pick.

The two methods (forward vs backward ports) are not mutually exclusive: on the
contrary, backporting commits that made it into master can make it easier for
the next person who wants to merge the stable branch into master.

Generally, we will encourage people to develop and test bug fixes against the
stable branch and forward port them by way of a merge into master. Due to the
obvious cases for exceptions, this will not be a "hard and fast" rule.

Documenting best practices considerations
==================================

There is a lot we need to document. The above needs to be refined, fleshed out
and turned into point-by-point documentation. There are many pages on Techbase
that need updating as a result of moving from svn to git as well. I'll
organize and hopefully also host a docu day on irc for this specifically once
the dust settles a bit more.

We'll need to have further discussion on these topics and your feedback is
welcome. Questions as well as answers are valuable, as both will help define
what needs to be further defined and documented.

Comments

Re: git workflow draft

By Jeremy Paul Whiting at 08/22/2011 - 12:33

Was this decided upon at some point? I got conflicting stories from
sysadmin and other developers. Yesterday after migrating kdeaccessibility
to git I was asked by a sysadmin to rename the X.Y branches to KDE/X.Y I
think concensus and consistency are important here. Was there a decision
that the official branches should be named X.Y? Is that documented
somewhere (I spent some time looking, but didn't find it). If not we should
reach concensus and also fix the repositories that are not following this
standard sooner than later imo. This will help greatly in the long run.

Thoughts, opinions, etc.
Jeremy

Re: git workflow draft

By Michael Pyne at 08/27/2011 - 20:27

Yes, this was decided (by a coin flip, natch) at a IRC meeting which was
preannounced on the various mailing lists.

The meeting itself was held 2011-Feb-13 at 18:00 (or so) U.S. EST. The
timestamp for the coin flip result was 18:51:29 on my logs (again, U.S. EST).

aseigo, bcooksley, PovAddict, Uninstall are the nearby names who could
probably attest if reminded.

My understanding was that someone was going to put the meeting results into a
Wiki (and it may have even happened ;), and that there would eventually be git
hooks implemented to ensure no KDE/x.y branches are created on official KDE
modules to make the rule self-documenting.

I'll continue to have only limited Internet connectivity until next week so I
hope this doesn't add fuel to some fire somewhere, but hopefully this helps. I
can gzip the logs I have an upload them somewhere if that's helpful.

Regards,
- Michael Pyne

Re: git workflow draft

By Michael Pyne at 08/27/2011 - 20:30

On Saturday, August 27, 2011 21:27:06 Michael Pyne wrote:
I see this was discussed more on kde-core-devel where support seems to be for
KDE/x.y.

Although I flipped the coin that gave us "x.y", I was voting even at the time
for "KDE/x.y" and fully support switching that "decision" if that's what we
end up doing.

Regards,
- Michael Pyne

Re: git workflow draft

By Aaron J. Seigo at 08/23/2011 - 01:15

On Monday, August 22, 2011 11:33:49 Jeremy Whiting wrote:
personally, i prefer the KDE/X.Y style as well; and as we haven't had more
accidently pushes of X.Y branches as people have become accustomed to the git
tools more, the original reason for suggesting to move away from KDE/X.Y to
just X.Y seems to have gone away?

not an official, consensus based on that i know of; there were some ad-hoc
decisions made, but that's about it.. and now we have some inconsistencies in
our modules. personally i like the KDE/X.Y branch names as they are
unambiguous and would like to use that style.

+1 to that.

my suggestion:

* give a 1 week period on this list for discussion and objections
* at the end of that period, if there is consensus, inform sysadmin and
request their opinion and consensus on the matter
* record this on techbase as part of our commit policies
* start fixing the modules that don't follow the consensus conclusion

Re: git workflow draft

By Rolf Eike Beer at 08/24/2011 - 01:06

Am Dienstag, 23. August 2011, 08:15:50 schrieb Aaron J. Seigo:
I also prefer the KDE/x.y scheme. The reason is rather simple: if a popular
application (let's say Amarok) which has a well-known version number that
differs from the KDE version number decides to release together with the KDE SC
one could simply do 2 tags: a.b for the application version and KDE/x.y for
the corresponding KDE SC version.

In fact I wonder if it wouldn't be a good idea to do this for things that are
part of the SC anyway, e.g. throw some tags for the KGpg version numbers in
that scheme too.

Eike

Re: git workflow draft

By Torgny Nyblom at 08/23/2011 - 23:20

On Tue, 23 Aug 2011 08:15:50 +0200, Aaron J. Seigo wrote:
My vote goes for the X.Y scheme as the repo is already under the KDE
namespace.
This was also the original scheme used that then was (accidentally ?)
not followed when more modules joined.

/Regards
Torgny

Re: git workflow draft

By Thomas Zander at 08/24/2011 - 06:29

On Wednesday 24 August 2011 06.20.06 Torgny Nyblom wrote:
That information is lost as soon as the repository is cloned, though.

As disc and bandwidth gets cheaper we'll probably see more mirror sites do
full mirrors of many open source git repositories from lots of different
sources, there this would get lost too.

I would be happier with the KDE/x.y solution, personally.

Re: git workflow draft

By Jeremy Paul Whiting at 08/24/2011 - 11:57

Since 4.7.1 tagging is a little over a week from now, I suggest we make a
decision between now and then. Then make the announcement on kcd,
release-team, etc. and put it on techbase, then a week later make the actual
branch renames. Sound good?

Jeremy

Re: git workflow draft

By Ben Cooksley at 08/23/2011 - 01:38

On Tue, Aug 23, 2011 at 6:15 PM, Aaron J. Seigo < ... at kde dot org> wrote:
I also prefer the KDE/x.y scheme - it is also used by more
repositories, so it will not be as disruptive to change to.

The accidental pushes are now blocked by git.kde.org if the branch
does not already exist. Branches of the form x.y or KDE/x.y (where x
and y are integers between 0-9) can only be created by repository
administrators.

(at least from my point of view as a Sysadmin, it doesn't matter which
scheme is chosen, so long as release-team is happy and it is
consistent across all KDE [SC] git modules).

+1

Regards,
Ben Cooksley
KDE Sysadmin

Re: git workflow draft

By Alexander Neundorf at 08/22/2011 - 14:32

On Monday 22 August 2011, Jeremy Whiting wrote:
My vote goes to "KDE/X.Y", it is clearer what it means.

Fully agree.

Fully agree.

Alex

Re: git workflow draft

By Stephen Kelly at 08/26/2011 - 05:06

When the frameworks get split out into multiple repos, will they still use
branch names KDE/5.0? What will that mean? Or will we come up with another
scheme then?

Thanks,

Steve.

Re: git workflow draft

By Sebastian =?utf... at 08/31/2011 - 06:00

On Friday, August 26, 2011 12:06:26 Stephen Kelly wrote:
I think they should, here's my reasoning:

The KDE part of the branch name means it's our "official" branch, i.e. it's
done by KDE (not by individuals acting on their own), so basically our
official namespace. We use $name/feature elsewhere, and I think KDE/5.37 would
blend in nicely here. On top of that, it gives some continuity.

Cheers,

Re: git workflow draft

By Alexander Neundorf at 09/12/2011 - 16:03

On Wednesday, August 31, 2011 01:00:56 PM Sebastian K├╝gler wrote:
Including the "5" ?
<a href="http://vizzzion.org/blog/2011/06/there-is-no-kde5/" title="http://vizzzion.org/blog/2011/06/there-is-no-kde5/">http://vizzzion.org/blog/2011/06/there-is-no-kde5/</a> ;-)

Alex

Re: git workflow draft

By Tsiapaliwkas Giorgos at 09/12/2011 - 18:58

I think that "5" has to stick around.When we go to KDE 6 how we will name it
if there isn't going to be any "5"?
And moreover there will be a lot of confusion then.

As i understand the sebas's blog post,it is irrelevant with the major
version number.This blog post means that there is not going to be a fifth
monolithic KDE platform.

Re: git workflow draft

By Jeremy Paul Whiting at 08/31/2011 - 10:07

Ok it seems most people with a preference prefer KDE/X.Y over X.Y and for
valid reasons
1) Other non-kde blessed branches can have obvious names.
2) Kdelibs, base, etc. are already KDE/X.Y
3) More modules are already KDE/X.Y than X.Y so less to fix when enforcing
consistency. (after looking at which repos would need to change, I'm not so
sure about this one...)

Thus I propose we agree to use KDE/X.Y for official kde release branches
going forward and after tagging tomorrow put a notice on relevant mailing
lists and on techbase. Then one week later perform the required changes to
existing repositories.
1 kdegraphics
2 kdeedu
3 kdeutils
4 kdepim
5 kdepim-runtime
6 kdeplasma-addons
7 kdepimlibs

If I've missed any in the above list, please add them.

best regards,
Jeremy

Re: git workflow draft

By Jeremy Paul Whiting at 08/31/2011 - 10:38

I forgot to mention some details about my proposal. See below.

My proposal to rename the branches is this. I (or someone else with access
to each repository) would checkout X.Y branch as KDE/X.Y then push it back
to the repository. Then the old X.Y branches on the repository would be
removed. This makes all the branches that are part of KDE SC follow the
consistent KDE/X.Y naming. Any existing clones with tracking branches to
X.Y named branches would then not be updated unfortunately. Which is the
reason for the announcement, posting the decision on techbase, including
instructions for updating tracking branches and so on.

In most cases, anyone with a clone without tracking branches will need to do
this:
git remote update
git remote prune origin (or whatever they named the remote)

In cases where tracking branches to X.Y branches are created, they will need
to be recreated to track KDE/X.Y instead.

I hope this clarifies my proposal a bit. Feel free to ask any questions.

Jeremy

Thus I propose we agree to use KDE/X.Y for official kde release branches

Re: git workflow draft

By Stephen Kelly at 02/16/2011 - 16:10

As one of the people asked to describe my idea of the workflow (which should
focus on rebasing, not merging) I put write up here:

<a href="http://community.kde.org/20110213_GitWorkflowAgenda/StevesIdea" title="http://community.kde.org/20110213_GitWorkflowAgenda/StevesIdea">http://community.kde.org/20110213_GitWorkflowAgenda/StevesIdea</a>

<a href="http://thread.gmane.org/gmane.comp.kde.scm-interest/2007" title="http://thread.gmane.org/gmane.comp.kde.scm-interest/2007">http://thread.gmane.org/gmane.comp.kde.scm-interest/2007</a>

The emphasis is on creating a 'nice' stream of commits which it is possible
to review (if applicable) and which apply against the target branch. This
way a reviewer (or archeologist looking for the source ofg a bug) doesn't
have to analyse commits in the branch which are workarounds for bugs in
master, find the merge from master and the removal of the workaround,
analyse the master branch in the same timeframe to see what needed to be
merged in etc etc.

The topic branch is an addition to the master branch. It should be relevant
against the current master and 'clean' and nice. Happy and good. No useless
merges. There should be no recommendation in documentation to 'merge a topic
branch into master'. The recommendation should be to rebase instead. This is
my recommendation. There has been some discussion on the scm interest list,
but I wouldn't call it concensus at this point.

The idea in the irc meeting was to not enable notifications and other such
hooks on the topic branches, but only on release branches.

That means there's only notification when master gets updated to include the
branch. That means you get a 100 commits becoming relevant within 10 seconds
instead of over the course of the 3 months or whatever that they were
written.

I fail to see why that would be a big deal. Reading the kde commits mailing
list is actually easier because I can skip a large amount of commits that
I'm not interested in at once.

All the best,

Steve.

Re: git workflow draft

By Johannes Sixt at 02/17/2011 - 03:01

Am 2/16/2011 22:10, schrieb Stephen Kelly:
Now I get it: The reason why you are advocating a rebase based workflow is
because the goal should be a series of self-contained, easy-to-review,
tested(!) commits.

That is, when someone (an individual or a group of collaborators) produces
a new feature, the result should be a (not too long) series of "clean"
commits. To achieve this, it is necessary to use git-rebase in order to
fold, split, and rearrange the commits. This sort of rebasing should
happen after a group of collaborators has produced a potentially messy
history with merges and fixups of earlier commits, and before the feature
nears publication to a wider audience. (You mentioned an example elsewhere
in this thread.)

So far, we are in the same boat, absolutely.

HOWEVER:

We disagree here: How a topic branch (that was prepared like above) ends
up in the master branch is an *entirely* different story.

When you develop a new feature, it is a very important choice on which
version of the software you base it on. I am advocating to base a feature
on a well-known, stable state. If you choose to base the feature on
"today's master", you are actually building a house on moving grounds. How
can you be sure that your product will be stable?

The corollary is that a new topic should be based on a project state that
is no younger than necessary (to have all prerequisites that you need), so
that you can be sure that it was already tested by a as many people as
possible. Ideally, today, every feature targeted for 4.7 should be based
on the 4.6.0 release tag, whenever possible.

As a consequence, you must not rebase the finished topic onto master, but
you must merge it.

-- Hannes

Re: git workflow draft

By Stephen Kelly at 02/17/2011 - 15:56

Correct.

Correct.

Correct.

That sounds like a workflow that is far too different from existing
workflows that KDE people use currently. It just won't be picked up. People
run master and develop against master. That's not going to change.

Additionally, you've made all history highly parallel instead of linear.
That makes trying to look into the history completely useless. Imagine
Alice, Bob, Charlie and Dave all wrote a feature which was interrelated to
the others enough that three of them had merge conflicts to solve. Now
imagine I have to go back in history to figure out what's going on. The
actual commits people made are valueless because they might have had to be
rewritten in a merge. So instead I have to look at the merges *and* all the
branches to try to figure out which branch won every conflict and what was
going on in the branch.

After a rebase no one else has to analyse a merge conflict and look at
certain commits while mentally ignoring them because they know the commit is
made irrelevant by a merge.

This is silly. When you merge in your workflow you have to re-run all tests
anyway.

I think this thread has run its course. Hopefully some people have learned
something about whether they should merge or rebase their topics.

Re: git workflow draft

By Johannes Sixt at 02/18/2011 - 03:21

I'm tired aguing, so I'll leave it at that. Just one point (because I
don't want to be called silly):

Am 2/17/2011 21:56, schrieb Stephen Kelly:
You are missing an important point here: When you merge, you run your
tests ONCE; when you rebase N commits you run your tests N TIMES!

-- Hannes

Re: git workflow draft

By Stephen Kelly at 02/18/2011 - 11:35

Just for clarity I wasn't calling you silly :).

I think we're just victims of low-bandwidth communication.

All the best,

Steve.

Re: git workflow draft

By Parker Coates at 02/18/2011 - 05:37

On Fri, Feb 18, 2011 at 03:21, Johannes Sixt wrote:
This is off topic, but is there a git tool to run a particular command
(for example, cmake && make && ./test) for every commit in a range?
Something like git-bisect without the bisecting.

More than once, I've rebased a local topic branch and been concerned
that some of the intermittent commits might no longer be buildable.
(This is especially an issue if commits are reordered.) Being able to
tell git to checkout and build each revision (halting if a build
fails) while I go do something else would be very convenient.

Parker

Re: git workflow draft

By Johannes Sixt at 02/18/2011 - 06:16

Am 2/18/2011 11:37, schrieb Parker Coates:
git rebase --interactive that-base-commit

and insert

x cmake && make && ./test

after each 'pick' line. (Since git 1.7.3.)

-- Hannes

Re: git workflow draft

By Wolfgang Rohdewald at 02/18/2011 - 05:55

On Freitag 18 Februar 2011, Parker Coates wrote:
for commit in `git log 64d2c1e...ba634b6 --pretty='format:%h'`
do
echo test $commit
done

Re: git workflow draft

By =?UTF-8?Q?Nicol... at 02/18/2011 - 14:15

On 18/02/2011, Wolfgang Rohdewald < ... at rohdewald dot de> wrote:
Better use "git rev-list 64d2c1e...ba634b6". And note ... and .. have
quite different meanings.

Re: git workflow draft

By Andreas Hartmetz at 02/17/2011 - 06:10

On Thursday 17 February 2011 09:01:05 Johannes Sixt wrote:
Maybe we'll find rules of thumb to choose the right base for topic branches.
Like "usually latest stable unless you rely on features of master" or "usually
master unless changes in master would interfere with your work"...
A checklist with obvious and not so obvious arguments for / against choosing a
particular base could also help make decisions on a case-by-case basis.

Does that sound reasonable or is it just my lack of git experience making me
suggest such things?

Re: git workflow draft

By Johannes Sixt at 02/17/2011 - 07:46

Am 2/17/2011 12:10, schrieb Andreas Hartmetz:
You do have a point here. However, I believe that it should not be a
problem in practice:

You start a topic from a stable state that is not today's master. You
develop the topic for a few days, or a week or two. During this time, you
do diligent testing, and at the end you are reasonably sure to have a new
good state. The goal here is that your testing does not suffer from
unrelated breakages introduced by other topics that are in flight.

At this point, you want to make sure the topic is good when it's merged
into master. Do that. And repeat all tests.

Now it's time to publish the topic. You do so by pushing the topic branch
(not the merge!). You encourage others to merge the topic into their
*private* repositories to get wider exposure and testing.

Yourself, you keep running the merged state, of course. You would also do
the regular merges from upstream that you used to do. See? You are still
testing master. If you like, you would merge topic branches offered by
others to test their topics as well.

When your topic has cooked sufficiently long, you would repeat the merge
into today's master (remember: do not expose your own private, messy merge
history to others), and push the result upstream. Now the topic is in master.

If the project hygenie that I sketched above were applied universally,
today's master would actually be a reasonable choice (contrary to what I
said earlier).

Until then, a relaxed rule is probably sufficient: Choose a starting point
that is convenient; but DO NOT CHANGE IT once you have done serious
development, because a change (aka rebase) basically invalidates all your
tests.

And common sense says: If you are cleaning up your messy topic history
using git-rebase in such a way that you have to re-test every new commit
anyway, you CAN choose a new starting point.

-- Hannes

Re: git workflow draft

By Matthias Fuchs at 02/19/2011 - 16:05

Am Donnerstag 17 Februar 2011, 13:46:04 schrieb Johannes Sixt:
I don't really agree.
Often you don't really depend on master, _but_ master will be the next stable
release.
There can be many interdependencies which would not show up if you based your
work on the current stable branch. And as a feature would never be added to
the stable branch anyway I see no reason to work on a branch of it for a
feature.
What would that gain me?

If I base it on master in fact errors could turn up which have nothing to do
with the feature. But would that be bad? Wouldn't that rather result in faster
fixes in bugs, no matter where they are? Wouldn't that result in a better next
stable version as errors turn up earlier?

Re: git workflow draft

By Stefan Majewsky at 02/16/2011 - 07:45

On Tue, Feb 15, 2011 at 6:51 PM, Aaron J. Seigo < ... at kde dot org> wrote:
As far as I'm concerned, the only problem with such branch moves is
the potentially epic number of commit notification mails. If so, the
email hook should check if the push generates a new branch, and send
only one mail then, like "100 commits have been pushed into the new
branch foobar; see >>here<< for a complete log vs. master and >>here<<
for the diff".

Greetings
Stefan

Re: git workflow draft

By Michael Pyne at 02/16/2011 - 16:05

On Wednesday, February 16, 2011 13:45:46 Stefan Majewsky wrote:
This actually came up elsewhere in the meeting as well.

There are two major proposed methods on integrating a feature development
branch back into master when ready.

1. You can use git-merge in order to merge 2 or more branches into a single
commit. The resulting commit is essentially just a normal commit with more
than one parent. Because of this, it is possible for git to verify that every
commit in history of each parent is present, and more importantly for this
problem, there is no duplication of commit emails (since there's only the 1
new commit).

A disadvantage is that it can clutter the output of GUI (and CLI) based tools,
especially if there are multiple active branches in development at once.

2. You can use git-rebase to basically reapply the diff for every commit in a
given branch off of a different base commit. For example:

* - A - G - H - I - J - K master
\ - B - C - D - E - F solid/make-it-cool

In this history tree make-it-cool and master have both diverged from A. If
git-rebase were run to integrate make-it-cool into master, the result would
look something like:

* - A - G - H - I - J - K - B' - C' - D' - E' - F' master

Every commit is still there. However, B', C', etc. are logically *new* commits
that were created by git-rebase, but with the same commit message (the commit
itself would be generated by generating the diff between the original commit
and its parent, and then re-applying on the new parent).

Since we're generating new commits anyways, it is also possible to "clean up"
the history upon integration (e.g. merge logically related commits C' and D'
into C', or split a too-large commit E' into E1' and E2').

However since they are new commits, they would also result in new commit
emails being sent, which could be quite the disaster for large branches. In
addition some of those re-sent commit emails might try to duplicate previously
taken actions for no reason (e.g. a needless CCMAIL: or GUI:).

An easy way to solve duplicates is to disable sending commit mails for
branches other than master, but I personally dislike that solution as it would
result in mailing lists like #kde-commits not receiving any emails until the
branch is fully landed on master. (I hate to say it, but imagine a security
mistake on ksslsocket.cpp in commit 88 of a 121-commit rebase branch -- is
everyone on #kde-commits going to review that deeply into the commit chain?)

There may be a way to allow emails on branches and still filter out duplicates
upon merging, but the sysadmins (IIRC) didn't know of a way to do it offhand.

In addition as Andreas Pakulat mentioned in a response to a rebase-workflow in
kde-scm-interest, this completely deletes the fact that the development
happened in a branch at all (we could simply retain the old solid/make-it-cool
ref so we don't lose that history, but that would make the repository
correspondingly larger).

A lot of this debate hinges on how we want email-based review to work however
(to be clear, kde-commits is *not* the only recipient of email for each git
commit, the CIA.vc web service also receives email for each commit). If we
want to avoid huge email bombs then we need to either use a merge-based
workflow in general, or have some other technical solution to allow for a
rebase-based workflow, such as your idea regarding lumping everything into a
simple notification email (although this means CIA.vc and possibly things like
Commit Filter wouldn't work... :( )

Regards,
- Michael Pyne

Re: git workflow draft

By Stephen Kelly at 02/16/2011 - 16:31

If someone writes 100 commits and pushes them without review then that's a
social problem. First of all I can't imagine a clean branch that could be so
big. Surely that would end up being a 10 commit push, some more work, a
push, some refactoring, a push, a feature, a push, more refactoring, a
feature, a push, etc. Do we really think that having large topic branches
which contain really only one topic that makes sense only to push in one go
will be a normal situation?

People who are interested in ksslsocket will see the commits.

The assumption that all of the history is useful to have by definition was
called into question. I suggested creating useful history so that all
history in the release branches is useful.

I guess we're writing emails at the same time here, but just to have it in
another place, I don't see 'email bombs' being a problem, but a convenient
grouping of stuff I'm interested in, and stuff I'm not interested in in
bigger lumps.

Another important point I think is that 'topic branches' will not
necessarily be normal, but exceptional. Most KDE developers are used to
commit early commit often, and that might translate into 'push early, push
often', so we'll end up seeing 2 or 3 commit pushes and very few monsters. I
rarely push more than 10 commits and very rarely if ever have pushed more
than 20.

Re: git workflow draft

By Stefan Majewsky at 02/17/2011 - 07:49

On Wed, Feb 16, 2011 at 10:31 PM, Stephen Kelly < ... at gmail dot com> wrote:
I was referring to the case when a feature branch gets moved between
repositories (e.g. a personal clone and the master repo). Review can
only happen when the feature branch is at a publicly visible place.

Greetings
Stefan

Re: git workflow draft

By Michael Pyne at 02/16/2011 - 17:36

On Wednesday, February 16, 2011 22:31:31 Stephen Kelly wrote:
That's actually a really good point, especially in terms of ways to avoid the
disadvantages I mention -- front-load the review for large branches just as we
already do with ReviewBoard.

You're thinking of CommitFilter. I'm thinking of the kde-commits mailing list
(i.e. people who didn't *know* they were interested in ksslsocket until they
saw a "strange" commit).

Either way is an assumption, but only one of these assumptions involves
deliberately discarding data. ;)

What specifically do you mean by "creating useful history" though? i.e. what
should be done additionally in a rebase workflow to get the useful history you
refer to?

I actually quite agree with you here for the vast majority of topic branches.
But it also mitigates one of the disadvantages I mentioned in merging (the
cluttered history), as there won't be too many active branches outstanding at
any given time, and therefore there shouldn't be large problems with tons of
branches at once anyways.

Regards,
- Michael Pyne

Re: git workflow draft

By Stephen Kelly at 02/16/2011 - 17:59

Michael Pyne wrote:
I wasn't really. It's just as true for kde-commits. People looking for
strange commits are not going to skip over commits just because they're in a
batch.

I know what you're going to say - They weren't looking for strange commits
until they found one :).

So why would anyone skip over a bunch of 15 commits in knetwork (or wherever
kssl is) just because they're all in one group? It seems to me like getting
into purely imaginary issues here.

Do this:

I think the most important thing to keep in mind in this whole git workflow
discussion is that the first iteration is not going to be correct no matter
what.

There will need to be changes and adjustments as we go along.

Re: git workflow draft

By Stephen Kelly at 02/16/2011 - 18:19

Sorry, knode fails me again. Some keyboard shortcut must be too close to
ctrl+v for me...

Stephen Kelly wrote:
If noise is data, you would have a good point.

cd /tmp
git clone git://gitorious.org/grantlee/grantlee.git
cd grantlee
git remote add mjansen git://gitorious.org/~mjansen/grantlee/mjansen-
grantlee.git
git fetch mjansen
gitk --all

Now look at the commits in the experimental branch from mjansen (before
v0.1.5).
See that some commits do lots of things at the same time/in the same commit
and should be split.

There's also commits for s/todo/TODO/, fixing copyrights etc. Having those
in separate commits is not 'useful historical data'.

There's also a workaround for a bug in the 0.1 branch which I eventually
fixed. The 0.1 branch was then merged into experimental and the workaround
removed. Speaking as the person who reviewed the branch, this and the large
commits issue made it harder to review. I had to navigate around the merges
in my head.

mjansen might just have been following a 'never rebase public branches'
philosphy, but that really doesn't work for me. It was a complicated feature
requiring lots of refactoring.

When issues crop up I want to know if it was refactoring that introduced the
issue, or the feature itself. I don't want to be looking back over the
commits 6 months from now, find the workaround, find the merge and removal
of the workaround, dig into the 0.1 branch to find out what the actual fix
was, find it wasn't relevant to the issue I'm trying to diagnose etc.

That's why the information conveyed by the 'actual history' is only noise
that increases maintenance burden and reviewability. If your topic branch
introduces something and then fixes it 5 commits later, then squash the fix
into the introduction and add a commment. A comment is a bazillion times
more visible than a hunk among many unrelated hunks in a large commit in a
long lived and then merged branch.

Look at the history of the actual repo between 0.1.6 and 0.1.7. Small
commits. First refactoring to make room for feature, then the feature. It
should be impossible to split a commit into two unrelated commits. At least
I try to stick to that in grantlee.

Re: git workflow draft

By Michael Pyne at 02/16/2011 - 20:03

On Thursday, February 17, 2011 00:19:56 Stephen Kelly wrote:
That's a great example, although I will point out that it is possible to have
fixed many of those in either workflow, by using git-rebase before pushing
fixup commits, commits that should be split, etc.

Having a rebase merge strategy does give you an additional chance to fix
commits since you're rewriting commits anyways, although that brings back the
email issue.

Of course as long as the number of commits involved is fairly small it sounds
like either way should work fine (and there's still the argument that people
are just going to do whatever anyways...), so it's probably worth documenting
how either workflow should proceed, and where it could be used, or should not
be used.

Regards,
- Michael Pyne

Re: git workflow draft

By Michael Jansen at 02/16/2011 - 18:51

Hehe ... as the one doing the code i would say it was more like ....

mjansen stumbled through unchartered territory with a blindfold and no clear
plan.

Or in other words. I knew what i wanted but not the means to do it. So i did
try and error while understanding your code. There is no way i could have
created a clean history for that branch unless doing it twice. Which more or
less happened with you beeing the one doing it twice.

Overall i concur with your email. It it just not always possible for mere
humans like me.

Mike

Re: git workflow draft

By Stephen Kelly at 02/16/2011 - 19:28

Yes, it's complicated code and a very complicated feature that I would never
have figured out. It didn't have to be done twice though. It was written
once and committed twice really.

The complicated nature of the feature is the reason for it, but the point I
was trying to make was that each of the noisy aspects should be discouraged
by discouraging merging and encuraging rebasing instead in the documented
workflows.

I don't want you to think I'm picking on your patches, it's just the set of
examples that I have closest to hand :).

Thanks,

Steve.

Re: git workflow draft

By Michael Jansen at 02/16/2011 - 21:55

A point where i btw. disagree. But i am not willing to go into a discussion
here. So in short. If you do a merge, merge. If you do a rebase, rebase. Both
have defined semantics and should be used accordingly.

If a feature branch is integrated it is a merge and should be done as that. I
expect a commit "Merged feature branch xyz.".

No offense taken.

Mike

Re: git workflow draft

By =?UTF-8?Q?Nicol... at 02/16/2011 - 18:10

On 16/02/2011, Stephen Kelly < ... at gmail dot com> wrote:
What happened to the content here? :)

Re: git workflow draft

By Ben Cooksley at 02/16/2011 - 16:47

On Thu, Feb 17, 2011 at 10:31 AM, Stephen Kelly < ... at gmail dot com> wrote:
Ah, you clearly have no understanding of the damage a "flood" or
"email bomb" causes.
Prior to git.kde.org being made available for mainstream use, in the
1st generation of hooks, a flood occurred.

This flood completely filled ktown's email queue, preventing any
kde.org email from being delivered until sysadmin destroyed the
offending emails from the queue. For the emails that were delivered,
it caused many bug reports to be altered, creating even more traffic,
and placed an enormous amount of stress upon CIA, which took many
hours for it to recover from, and blocked notifications for all other
open source projects which use CIA for several hours.

The effects of an email bomb can be extremely dangerous. They cannot
be permitted to occur under any circumstances.

Regards,
Ben Cooksley
KDE Sysadmin

Re: git workflow draft

By Stephen Kelly at 02/16/2011 - 17:51

Ben Cooksley wrote:
Correct :)

Ok, thanks for educating me there.

It sounds like what is needed is throttling built into the git hook. Is that
possible?

while (!availableCommits.isEmpty()) {
// process 15 commits
availableCommits = dispatch(availableCommits, 15);
sleep(30 seconds);
}

Re: git workflow draft

By John Layt at 02/15/2011 - 13:32

On Tuesday 15 February 2011 17:51:35 Aaron J. Seigo wrote:
Actually, even more complete minutes, outstanding issues and action points can
be found at <a href="http://community.kde.org/20110213_GitWorkflowAgenda" title="http://community.kde.org/20110213_GitWorkflowAgenda">http://community.kde.org/20110213_GitWorkflowAgenda</a>

The IRC log can be found at <a href="http://community.kde.org/20110213_GitWorkflowLog" title="http://community.kde.org/20110213_GitWorkflowLog">http://community.kde.org/20110213_GitWorkflowLog</a>

John.

Re: git workflow draft

By John Layt at 02/16/2011 - 07:58

On Tuesday 15 February 2011 18:32:29 John Layt wrote:
Following up on my action points for the commit template and config settings.

I've attached a revised commit template below. The suggestion is for this to
be added to each project repo in the root folder to allow for easy
distribution and updating. It would also allow each project to customise it
as required, e.g. add links to projects own commit policy, extra tags, etc.

With regards to recommended git config settings, we could set the repo
defaults in .git/config to include things like the kde: shortcuts, the commit
template, and possibly push.default none. Do we want to do this? It would
mean fewer steps required by a dev to set up their environment, and they can
be changed by more skilled devs who don't want them (I'm assuming a locally
modified .git/config doesn't get updated from the remote after the initial
clone). Any drawbacks?

I want to make a start on some of the Git recipe and reference documentation
as things occur to me, and was thinking a central
<a href="http://techbase.kde.org/Development/Git" title="http://techbase.kde.org/Development/Git">http://techbase.kde.org/Development/Git</a> hub page leading off to the various
tutorial, policy, recipe, sysadmin, etc pages would be a good idea. Sound OK?

Cheers!

John.

# ---[ You MUST wrap all lines at 72 characters ]----------------------|
#
# ---[ Please see <a href="http://techbase.kde.org/Policies/Commit_Policy" title="http://techbase.kde.org/Policies/Commit_Policy">http://techbase.kde.org/Policies/Commit_Policy</a> ]-----|
#
# ===[ Subject ]=======================================================|
# ---[ One line only, short meaningful description to show in logs ]---|

# ---[ Leave following line blank, do NOT remove ]---------------------|

# ===[ Details ]=======================================================|
# ---[ Describe what has changed and explain why it has changed ]------|

# ===[ Fields ]========================================================|
# ---[ Uncomment and edit as applicable ]------------------------------|

# ---[ Add Feature to release changelog ]
# ---[ Optionally close a wish in bugs.kde.org as implemented ]
#FEATURE: <optional bug number>
#
# ---[ Close a bug in bugs.kde.org as fixed, with optional version ]
#BUG: <bug number>
#FIXED-IN: <release version>
#
# ---[ Copy commit message to a bug in bugs.kde.org ]
#CCBUG: <bug number>
#
# ---[ Copy commit message to an email address ]
#CCMAIL: <email>
#
# ---[ Close a review in reviewboard.kde.org as submitted ]
#REVIEW: <review number>
#
# ---[ Advise documentation team of user visible changes in the gui ]
#GUI:
#
# =====================================================================|

Re: git workflow draft

By John Layt at 02/17/2011 - 15:36

On Wednesday 16 February 2011 12:58:48 John Layt wrote:
OK, I've created <a href="http://techbase.kde.org/Development/Git" title="http://techbase.kde.org/Development/Git">http://techbase.kde.org/Development/Git</a> and started
organising stuff from there. I've added a list of all the existing svn and
git pages in techbase and community to start working through revising.

John.

Re: git workflow draft

By Raphael Kubo da... at 02/16/2011 - 09:05

On Wednesday 16 February 2011 10:58:48 John Layt wrote:
Personally, I find that starting this short description with the "module",
"library" or whatever being touched helps when one is skimming through the
log. "kzip: Fix typo" or "kdecore: Fix typo" looks better than "Fix typo".

Is it worth mentioning in the template?

Re: git workflow draft

By Michael Pyne at 02/16/2011 - 15:40

On Wednesday, February 16, 2011 12:05:51 Raphael Kubo da Costa wrote:
I would say yes. It matches existing convention from what I can tell, plus
it's simply a good idea as it makes things like git log --oneline a bit more
useful to try to grep through.

At the same time it's not necessarily a big deal if it's left out, as you can
always manually pass a path to git log to have it return only log entries for
the path you're interested in.

Regards,
- Michael Pyne