forked from python/peps
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathpep-0374.txt
1518 lines (1203 loc) · 53.1 KB
/
pep-0374.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
PEP: 374
Title: Choosing a distributed VCS for the Python project
Version: $Revision$
Last-Modified: $Date$
Author: Brett Cannon <[email protected]>,
Stephen J. Turnbull <[email protected]>,
Alexandre Vassalotti <[email protected]>,
Barry Warsaw <[email protected]>,
Dirkjan Ochtman <[email protected]>
Status: Final
Type: Process
Content-Type: text/x-rst
Created: 07-Nov-2008
Post-History: 07-Nov-2008
22-Jan-2009
Rationale
=========
Python has been using a centralized version control system (VCS;
first CVS, now Subversion) for years to great effect. Having a master
copy of the official version of Python provides people with a single
place to always get the official Python source code. It has also
allowed for the storage of the history of the language, mostly for
help with development, but also for posterity. And of course the V in
VCS is very helpful when developing.
But a centralized version control system has its drawbacks. First and
foremost, in order to have the benefits of version control with
Python in a seamless fashion, one must be a "core developer" (i.e.
someone with commit privileges on the master copy of Python). People
who are not core developers but who wish to work with Python's
revision tree, e.g. anyone writing a patch for Python or creating a
custom version, do not have direct tool support for revisions. This
can be quite a limitation, since these non-core developers cannot
easily do basic tasks such as reverting changes to a previously
saved state, creating branches, publishing one's changes with full
revision history, etc. For non-core developers, the last safe tree
state is one the Python developers happen to set, and this prevents
safe development. This second-class citizenship is a hindrance to
people who wish to contribute to Python with a patch of any
complexity and want a way to incrementally save their progress to
make their development lives easier.
There is also the issue of having to be online to be able to commit
one's work. Because centralized VCSs keep a central copy that stores
all revisions, one must have Internet access in order for their
revisions to be stored; no Net, no commit. This can be annoying if
you happen to be traveling and lack any Internet. There is also the
situation of someone wishing to contribute to Python but having a
bad Internet connection where committing is time-consuming and
expensive and it might work out better to do it in a single step.
Another drawback to a centralized VCS is that a common use case is
for a developer to revise patches in response to review comments.
This is more difficult with a centralized model because there's no
place to contain intermediate work. It's either all checked in or
none of it is checked in. In the centralized VCS, it's also very
difficult to track changes to the trunk as they are committed, while
you're working on your feature or bug fix branch. This increases
the risk that such branches will grow stale, out-dated, or that
merging them into the trunk will generate too may conflicts to be
easily resolved.
Lastly, there is the issue of maintenance of Python. At any one time
there is at least one major version of Python under development (at
the time of this writing there are two). For each major version of
Python under development there is at least the maintenance version
of the last minor version and the in-development minor version (e.g.
with 2.6 just released, that means that both 2.6 and 2.7 are being
worked on). Once a release is done, a branch is created between the
code bases where changes in one version do not (but could) belong in
the other version. As of right now there is no natural support for
this branch in time in central VCSs; you must use tools that
simulate the branching. Tracking merges is similarly painful for
developers, as revisions often need to be merged between four active
branches (e.g. 2.6 maintenance, 3.0 maintenance, 2.7 development,
3.1 development). In this case, VCSs such as Subversion only handle
this through arcane third party tools.
Distributed VCSs (DVCSs) solve all of these problems. While one can
keep a master copy of a revision tree, anyone is free to copy that
tree for their own use. This gives everyone the power to commit
changes to their copy, online or offline. It also more naturally
ties into the idea of branching in the history of a revision tree
for maintenance and the development of new features bound for
Python. DVCSs also provide a great many additional features that
centralized VCSs don't or can't provide.
This PEP explores the possibility of changing Python's use of Subversion
to any of the currently popular DVCSs, in order to gain
the benefits outlined above. This PEP does not guarantee that a switch
to a DVCS will occur at the conclusion of this PEP. It is quite
possible that no clear winner will be found and that svn will continue
to be used. If this happens, this PEP will be revisited and revised in
the future as the state of DVCSs evolves.
Terminology
===========
Agreeing on a common terminology is surprisingly difficult,
primarily because each VCS uses these terms when describing subtly
different tasks, objects, and concepts. Where possible, we try to
provide a generic definition of the concepts, but you should consult
the individual system's glossaries for details. Here are some basic
references for terminology, from some of the standard web-based
references on each VCS. You can also refer to glossaries for each
DVCS:
* Subversion : http://svnbook.red-bean.com/en/1.5/svn.basic.html
* Bazaar : http://bazaar-vcs.org/BzrGlossary
* Mercurial : http://www.selenic.com/mercurial/wiki/index.cgi/UnderstandingMercurial
* git : http://book.git-scm.com/1_the_git_object_model.html
branch
A line of development; a collection of revisions, ordered by
time.
checkout/working copy/working tree
A tree of code the developer can edit, linked to a branch.
index
A "staging area" where a revision is built (unique to git).
repository
A collection of revisions, organized into branches.
clone
A complete copy of a branch or repository.
commit
To record a revision in a repository.
merge
Applying all the changes and history from one branch/repository
to another.
pull
To update a checkout/clone from the original branch/repository,
which can be remote or local
push/publish
To copy a revision, and all revisions it depends on, from a one
repository to another.
cherry-pick
To merge one or more specific revisions from one branch to
another, possibly in a different repository, possibly without its
dependent revisions.
rebase
To "detach" a branch, and move it to a new branch point; move
commits to the beginning of a branch instead of where they
happened in time.
Typical Workflow
================
At the moment, the typical workflow for a Python core developer is:
* Edit code in a checkout until it is stable enough to commit/push.
* Commit to the master repository.
It is a rather simple workflow, but it has drawbacks. For one,
because any work that involves the repository takes time thanks to
the network, commits/pushes tend to not necessarily be as atomic as
possible. There is also the drawback of there not being a
necessarily cheap way to create new checkouts beyond a recursive
copy of the checkout directory.
A DVCS would lead to a workflow more like this:
* Branch off of a local clone of the master repository.
* Edit code, committing in atomic pieces.
* Merge the branch into the mainline, and
* Push all commits to the master repository.
While there are more possible steps, the workflow is much more
independent of the master repository than is currently possible. By
being able to commit locally at the speed of your disk, a core
developer is able to do atomic commits much more frequently,
minimizing having commits that do multiple things to the code. Also
by using a branch, the changes are isolated (if desired) from other
changes being made by other developers. Because branches are cheap,
it is easy to create and maintain many smaller branches that address
one specific issue, e.g. one bug or one new feature. More
sophisticated features of DVCSs allow the developer to more easily
track long running development branches as the official mainline
progresses.
Contenders
==========
========== ========== ======= =================================== ==========================================
Name Short Name Version 2.x Trunk Mirror 3.x Trunk Mirror
========== ========== ======= =================================== ==========================================
Bazaar_ bzr 1.12 http://code.python.org/python/trunk http://code.python.org/python/3.0
Mercurial_ hg 1.2.0 http://code.python.org/hg/trunk/ http://code.python.org/hg/branches/py3k/
git_ N/A 1.6.1 git://code.python.org/python/trunk git://code.python.org/python/branches/py3k
========== ========== ======= =================================== ==========================================
.. _Bazaar: http://bazaar-vcs.org/
.. _Mercurial: http://www.selenic.com/mercurial/
.. _git: http://www.git-scm.com/
This PEP does not consider darcs, arch, or monotone. The main
problem with these DVCSs is that they are simply not popular enough
to bother supporting when they do not provide some very compelling
features that the other DVCSs provide. Arch and darcs also have
significant performance problems which seem unlikely to be addressed
in the near future.
Interoperability
================
For those who have already decided which DVCSs they want to use, and
are willing to maintain local mirrors themselves, all three DVCSs
support interchange via the git "fast-import" changeset format. git
does so natively, of course, and native support for Bazaar is under
active development, and getting good early reviews as of mid-February
2009. Mercurial has idiosyncratic support for importing via its *hg
convert* command, and `third-party fast-import support`_ is available
for exporting. Also, the Tailor_ tool supports automatic maintenance
of mirrors based on an official repository in any of the candidate
formats with a local mirror in any format.
.. _third-party fast-import support: http://repo.or.cz/r/fast-export.git/.git/description
.. _Tailor: http://progetti.arstecnica.it/tailor/
Usage Scenarios
===============
Probably the best way to help decide on whether/which DVCS should
replace Subversion is to see what it takes to perform some
real-world usage scenarios that developers (core and non-core) have
to work with. Each usage scenario outlines what it is, a bullet list
of what the basic steps are (which can vary slightly per VCS), and
how to perform the usage scenario in the various VCSs
(including Subversion).
Each VCS had a single author in charge of writing implementations
for each scenario (unless otherwise noted).
========= ===
Name VCS
========= ===
Brett svn
Barry bzr
Alexandre hg
Stephen git
========= ===
Initial Setup
-------------
Some DVCSs have some perks if you do some initial setup upfront.
This section covers what can be done before any of the usage
scenarios are run in order to take better advantage of the tools.
All of the DVCSs support configuring your project identification.
Unlike the centralized systems, they use your email address to
identify your commits. (Access control is generally done by
mechanisms external to the DVCS, such as ssh or console login).
This identity may be associated with a full name.
All of the DVCSs will query the system to get some approximation to
this information, but that may not be what you want. They also
support setting this information on a per-user basis, and on a per-
project basis. Convenience commands to set these attributes vary,
but all allow direct editing of configuration files.
Some VCSs support end-of-line (EOL) conversions on checkout/checkin.
svn
'''
None required, but it is recommended you follow the
`guidelines <http://www.python.org/dev/faq/#what-configuration-settings-should-i-use>`_
in the dev FAQ.
bzr
'''
No setup is required, but for much quicker and space-efficient local
branching, you should create a shared repository to hold all your
Python branches. A shared repository is really just a parent
directory containing a .bzr directory. When bzr commits a revision,
it searches from the local directory on up the file system for a .bzr
directory to hold the revision. By sharing revisions across multiple
branches, you cut down on the amount of disk space used. Do this::
cd ~/projects
bzr init-repo python
cd python
Now, all your Python branches should be created inside of
``~/projects/python``.
There are also some settings you can put in your
``~/.bzr/bazaar.conf``
and ``~/.bzr/locations.conf`` file to set up defaults for interacting
with Python code. None of them are required, although some are
recommended. E.g. I would suggest gpg signing all commits, but that
might be too high a barrier for developers. Also, you can set up
default push locations depending on where you want to push branches
by default. If you have write access to the master branches, that
push location could be code.python.org. Otherwise, it might be a
free Bazaar code hosting service such as Launchpad. If Bazaar is
chosen, we should decide what the policies and recommendations are.
At a minimum, I would set up your email address::
bzr whoami "Firstname Lastname <[email protected]>"
As with hg and git below, there are ways to set your email address (or really,
just about any parameter) on a
per-repository basis. You do this with settings in your
``$HOME/.bazaar/locations.conf`` file, which has an ini-style format as does
the other DVCSs. See the Bazaar documentation for details,
which mostly aren't relevant for this discussion.
hg
''
Minimally, you should set your user name. To do so, create the file
``.hgrc`` in your home directory and add the following::
[ui]
username = Firstname Lastname <[email protected]>
If you are using Windows and your tools do not support Unix-style newlines,
you can enable automatic newline translation by adding to your configuration::
[extensions]
win32text =
These options can also be set locally to a given repository by
customizing ``<repo>/.hg/hgrc``, instead of ``~/.hgrc``.
git
'''
None needed. However, git supports a number of features that can
smooth your work, with a little preparation. git supports setting
defaults at the workspace, user, and system levels. The system
level is out of scope of this PEP. The user configuration file is
``$HOME/.gitconfig`` on Unix-like systems, and the workspace
configuration file is ``$REPOSITORY/.git/config``.
You can use the ``git-config`` tool to set preferences for user.name and
user.email either globally (for your system login account) or
locally (to a given git working copy), or you can edit the
configuration files (which have the same format as shown in the
Mercurial section above).::
# my full name doesn't change
# note "--global" flag means per user
# (system-wide configuration is set with "--system")
git config --global user.name 'Firstname Lastname'
# but use my Pythonic email address
cd /path/to/python/repository
git config user.email [email protected]
If you are using Windows, you probably want to set the core.autocrlf
and core.safecrlf preferences to true using ``git-config``.::
# check out files with CRLF line endings rather than Unix-style LF only
git config --global core.autocrlf true
# scream if a transformation would be ambiguous
# (eg, a working file contains both naked LF and CRLF)
# and check them back in with the reverse transformation
git config --global core.safecrlf true
Although the repository will usually contain a .gitignore file
specifying file names that rarely if ever should be registered in the
VCS, you may have personal conventions (e.g., always editing log
messages in a temporary file named ".msg") that you may wish to
specify.::
# tell git where my personal ignores are
git config --global core.excludesfile ~/.gitignore
# I use .msg for my long commit logs, and Emacs makes backups in
# files ending with ~
# these are globs, not regular expressions
echo '*~' >> ~/.gitignore
echo '.msg' >> ~/.gitignore
If you use multiple branches, as with the other VCSes, you can save a
lot of space by putting all objects in a common object store. This
also can save download time, if the origins of the branches were in
different repositories, because objects are shared across branches in
your repository even if they were not present in the upstream
repositories. git is very space- and time-efficient and applies a
number of optimizations automatically, so this configuration is
optional. (Examples are omitted.)
One-Off Checkout
----------------
As a non-core developer, I want to create and publish a one-off patch
that fixes a bug, so that a core developer can review it for
inclusion in the mainline.
* Checkout/branch/clone trunk.
* Edit some code.
* Generate a patch (based on what is best supported by the VCS, e.g.
branch history).
* Receive reviewer comments and address the issues.
* Generate a second patch for the core developer to commit.
svn
'''
::
svn checkout http://svn.python.org/projects/python/trunk
cd trunk
# Edit some code.
echo "The cake is a lie!" > README
# Since svn lacks support for local commits, we fake it with patches.
svn diff >> commit-1.diff
svn diff >> patch-1.diff
# Upload the patch-1 to bugs.python.org.
# Receive reviewer comments.
# Edit some code.
echo "The cake is real!" > README
# Since svn lacks support for local commits, we fake it with patches.
svn diff >> commit-2.diff
svn diff >> patch-2.diff
# Upload patch-2 to bugs.python.org
bzr
'''
::
bzr branch http://code.python.org/python/trunk
cd trunk
# Edit some code.
bzr commit -m 'Stuff I did'
bzr send -o bundle
# Upload bundle to bugs.python.org
# Receive reviewer comments
# Edit some code
bzr commit -m 'Respond to reviewer comments'
bzr send -o bundle
# Upload updated bundle to bugs.python.org
The ``bundle`` file is like a super-patch. It can be read by ``patch(1)`` but
it contains additional metadata so that it can be fed to ``bzr merge`` to
produce a fully usable branch completely with history. See `Patch Review`_
section below.
hg
''
::
hg clone http://code.python.org/hg/trunk
cd trunk
# Edit some code.
hg commit -m "Stuff I did"
hg outgoing -p > fixes.patch
# Upload patch to bugs.python.org
# Receive reviewer comments
# Edit some code
hg commit -m "Address reviewer comments."
hg outgoing -p > additional-fixes.patch
# Upload patch to bugs.python.org
While ``hg outgoing`` does not have the flag for it, most Mercurial
commands support git's extended patch format through a ``--git``
command. This can be set in one's ``.hgrc`` file so that all commands
that generate a patch use the extended format.
git
'''
The patches could be created with
``git diff master > stuff-i-did.patch``, too, but
``git format-patch | git am`` knows some tricks
(empty files, renames, etc) that ordinary patch can't handle. git
grabs "Stuff I did" out of the commit message to create the file
name 0001-Stuff-I-did.patch. See Patch Review below for a
description of the git-format-patch format.
::
# Get the mainline code.
git clone git://code.python.org/python/trunk
cd trunk
# Edit some code.
git commit -a -m 'Stuff I did.'
# Create patch for my changes (i.e, relative to master).
git format-patch master
git tag stuff-v1
# Upload 0001-Stuff-I-did.patch to bugs.python.org.
# Time passes ... receive reviewer comments.
# Edit more code.
git commit -a -m 'Address reviewer comments.'
# Make an add-on patch to apply on top of the original.
git format-patch stuff-v1
# Upload 0001-Address-reviewer-comments.patch to bugs.python.org.
Backing Out Changes
-------------------
As a core developer, I want to undo a change that was not ready for
inclusion in the mainline.
* Back out the unwanted change.
* Push patch to server.
svn
'''
::
# Assume the change to revert is in revision 40
svn merge -c -40 .
# Resolve conflicts, if any.
svn commit -m "Reverted revision 40"
bzr
'''
::
# Assume the change to revert is in revision 40
bzr merge -r 40..39
# Resolve conflicts, if any.
bzr commit -m "Reverted revision 40"
Note that if the change you want revert is the last one that was
made, you can just use ``bzr uncommit``.
hg
''
::
# Assume the change to revert is in revision 9150dd9c6d30
hg backout --merge -r 9150dd9c6d30
# Resolve conflicts, if any.
hg commit -m "Reverted changeset 9150dd9c6d30"
hg push
Note, you can use "hg rollback" and "hg strip" to revert changes you committed
in your local repository, but did not yet push to other repositories.
git
'''
::
# Assume the change to revert is the grandfather of a revision tagged "newhotness".
git revert newhotness~2
# Resolve conflicts if any. If there are no conflicts, the commit
# will be done automatically by "git revert", which prompts for a log.
git commit -m "Reverted changeset 9150dd9c6d30."
git push
Patch Review
------------
As a core developer, I want to review patches submitted by other
people, so that I can make sure that only approved changes are added
to Python.
Core developers have to review patches as submitted by other people.
This requires applying the patch, testing it, and then tossing away
the changes. The assumption can be made that a core developer already
has a checkout/branch/clone of the trunk.
* Branch off of trunk.
* Apply patch w/o any comments as generated by the patch submitter.
* Push patch to server.
* Delete now-useless branch.
svn
'''
Subversion does not exactly fit into this development style very well
as there are no such thing as a "branch" as has been defined in this
PEP. Instead a developer either needs to create another checkout for
testing a patch or create a branch on the server. Up to this point,
core developers have not taken the "branch on the server" approach to
dealing with individual patches. For this scenario the assumption
will be the developer creates a local checkout of the trunk to work
with.::
cp -r trunk issue0000
cd issue0000
patch -p0 < __patch__
# Review patch.
svn commit -m "Some patch."
cd ..
rm -r issue0000
Another option is to only have a single checkout running at any one
time and use ``svn diff`` along with ``svn revert -R`` to store away
independent changes you may have made.
bzr
'''
::
bzr branch trunk issueNNNN
# Download `patch` bundle from Roundup
bzr merge patch
# Review patch
bzr commit -m'Patch NNN by So N. So' --fixes python:NNNN
bzr push bzr+ssh://[email protected]/trunk
rm -rf ../issueNNNN
Alternatively, since you're probably going to commit these changes to
the trunk, you could just do a checkout. That would give you a local
working tree while the branch (i.e. all revisions) would continue to
live on the server. This is similar to the svn model and might allow
you to more quickly review the patch. There's no need for the push
in this case.::
bzr checkout trunk issueNNNN
# Download `patch` bundle from Roundup
bzr merge patch
# Review patch
bzr commit -m'Patch NNNN by So N. So' --fixes python:NNNN
rm -rf ../issueNNNN
hg
''
::
hg clone trunk issue0000
cd issue0000
# If the patch was generated using hg export, the user name of the
# submitter is automatically recorded. Otherwise,
# use hg import --no-commit submitted.diff and commit with
# hg commit -u "Firstname Lastname <[email protected]>"
hg import submitted.diff
# Review patch.
hg push ssh://[email protected]/hg/trunk/
git
'''
We assume a patch created by git-format-patch. This is a Unix mbox
file containing one or more patches, each formatted as an RFC 2822
message. git-am interprets each message as a commit as follows. The
author of the patch is taken from the From: header, the date from the
Date header. The commit log is created by concatenating the content
of the subject line, a blank line, and the message body up to the
start of the patch.::
cd trunk
# Create a branch in case we don't like the patch.
# This checkout takes zero time, since the workspace is left in
# the same state as the master branch.
git checkout -b patch-review
# Download patch from bugs.python.org to submitted.patch.
git am < submitted.patch
# Review and approve patch.
# Merge into master and push.
git checkout master
git merge patch-review
git push
Backport
--------
As a core developer, I want to apply a patch to 2.6, 2.7, 3.0, and 3.1
so that I can fix a problem in all three versions.
Thanks to always having the cutting-edge and the latest release
version under development, Python currently has four branches being
worked on simultaneously. That makes it important for a change to
propagate easily through various branches.
svn
'''
Because of Python's use of svnmerge, changes start with the trunk
(2.7) and then get merged to the release version of 2.6. To get the
change into the 3.x series, the change is merged into 3.1, fixed up,
and then merged into 3.0 (2.7 -> 2.6; 2.7 -> 3.1 -> 3.0).
This is in contrast to a port-forward strategy where the patch would
have been added to 2.6 and then pulled forward into newer versions
(2.6 -> 2.7 -> 3.0 -> 3.1).
::
# Assume patch applied to 2.7 in revision 0000.
cd release26-maint
svnmerge merge -r 0000
# Resolve merge conflicts and make sure patch works.
svn commit -F svnmerge-commit-message.txt # revision 0001.
cd ../py3k
svnmerge merge -r 0000
# Same as for 2.6, except Misc/NEWS changes are reverted.
svn revert Misc/NEWS
svn commit -F svnmerge-commit-message.txt # revision 0002.
cd ../release30-maint
svnmerge merge -r 0002
svn commit -F svnmerge-commit-message.txt # revision 0003.
bzr
'''
Bazaar is pretty straightforward here, since it supports cherry
picking revisions manually. In the example below, we could have
given a revision id instead of a revision number, but that's usually
not necessary. Martin Pool suggests "We'd generally recommend doing
the fix first in the oldest supported branch, and then merging it
forward to the later releases."::
# Assume patch applied to 2.7 in revision 0000
cd release26-maint
bzr merge ../trunk -c 0000
# Resolve conflicts and make sure patch works
bzr commit -m 'Back port patch NNNN'
bzr push bzr+ssh://[email protected]/trunk
cd ../py3k
bzr merge ../trunk -r 0000
# Same as for 2.6 except Misc/NEWS changes are reverted
bzr revert Misc/NEWS
bzr commit -m 'Forward port patch NNNN'
bzr push bzr+ssh://[email protected]/py3k
hg
''
Mercurial, like other DVCS, does not well support the current
workflow used by Python core developers to backport patches. Right
now, bug fixes are first applied to the development mainline
(i.e., trunk), then back-ported to the maintenance branches and
forward-ported, as necessary, to the py3k branch. This workflow
requires the ability to cherry-pick individual changes. Mercurial's
transplant extension provides this ability. Here is an example of
the scenario using this workflow::
cd release26-maint
# Assume patch applied to 2.7 in revision 0000
hg transplant -s ../trunk 0000
# Resolve conflicts, if any.
cd ../py3k
hg pull ../trunk
hg merge
hg revert Misc/NEWS
hg commit -m "Merged trunk"
hg push
In the above example, transplant acts much like the current svnmerge
command. When transplant is invoked without the revision, the command
launches an interactive loop useful for transplanting multiple
changes. Another useful feature is the --filter option which can be
used to modify changesets programmatically (e.g., it could be used
for removing changes to Misc/NEWS automatically).
Alternatively to the traditional workflow, we could avoid
transplanting changesets by committing bug fixes to the oldest
supported release, then merge these fixes upward to the more recent
branches.
::
cd release25-maint
hg import fix_some_bug.diff
# Review patch and run test suite. Revert if failure.
hg push
cd ../release26-maint
hg pull ../release25-maint
hg merge
# Resolve conflicts, if any. Then, review patch and run test suite.
hg commit -m "Merged patches from release25-maint."
hg push
cd ../trunk
hg pull ../release26-maint
hg merge
# Resolve conflicts, if any, then review.
hg commit -m "Merged patches from release26-maint."
hg push
Although this approach makes the history non-linear and slightly
more difficult to follow, it encourages fixing bugs across all
supported releases. Furthermore, it scales better when there is many
changes to backport, because we do not need to seek the specific
revision IDs to merge.
git
'''
In git I would have a workspace which contains all of
the relevant master repository branches. git cherry-pick doesn't
work across repositories; you need to have the branches in the same
repository.
::
# Assume patch applied to 2.7 in revision release27~3 (4th patch back from tip).
cd integration
git checkout release26
git cherry-pick release27~3
# If there are conflicts, resolve them, and commit those changes.
# git commit -a -m "Resolve conflicts."
# Run test suite. If fixes are necessary, record as a separate commit.
# git commit -a -m "Fix code causing test failures."
git checkout master
git cherry-pick release27~3
# Do any conflict resolution and test failure fixups.
# Revert Misc/NEWS changes.
git checkout HEAD^ -- Misc/NEWS
git commit -m 'Revert cherry-picked Misc/NEWS changes.' Misc/NEWS
# Push both ports.
git push release26 master
If you are regularly merging (rather than cherry-picking) from a
given branch, then you can block a given commit from being
accidentally merged in the future by merging, then reverting it.
This does not prevent a cherry-pick from pulling in the unwanted
patch, and this technique requires blocking everything that you don't
want merged. I'm not sure if this differs from svn on this point.
::
cd trunk
# Merge in the alpha tested code.
git merge experimental-branch
# We don't want the 3rd-to-last commit from the experimental-branch,
# and we don't want it to ever be merged.
# The notation "^N" means Nth parent of the current commit. Thus HEAD^2^1^1
# means the first parent of the first parent of the second parent of HEAD.
git revert HEAD^2^1^1
# Propagate the merge and the prohibition to the public repository.
git push
Coordinated Development of a New Feature
----------------------------------------
Sometimes core developers end up working on a major feature with
several developers. As a core developer, I want to be able to
publish feature branches to a common public location so that I can
collaborate with other developers.
This requires creating a branch on a server that other developers
can access. All of the DVCSs support creating new repositories on
hosts where the developer is already able to commit, with
appropriate configuration of the repository host. This is
similar in concept to the existing sandbox in svn, although details
of repository initialization may differ.
For non-core developers, there are various more-or-less public-access
repository-hosting services.
Bazaar has
Launchpad_,
Mercurial has
`bitbucket.org`_,
and git has
GitHub_.
All also have easy-to-use
CGI interfaces for developers who maintain their own servers.
.. _Launchpad: http://www.launchpad.net/
.. _bitbucket.org: http://www.bitbucket.org/
.. _GitHub: http://www.github.com/
* Branch trunk.
* Pull from branch on the server.
* Pull from trunk.
* Push merge to trunk.
svn
'''
::
# Create branch.
svn copy svn+ssh://[email protected]/python/trunk svn+ssh://[email protected]/python/branches/NewHotness
svn checkout svn+ssh://[email protected]/python/branches/NewHotness
cd NewHotness
svnmerge init
svn commit -m "Initialize svnmerge."
# Pull in changes from other developers.
svn update
# Pull in trunk and merge to the branch.
svnmerge merge
svn commit -F svnmerge-commit-message.txt
This scenario is incomplete as the decision for what DVCS to go with
was made before the work was complete.
Separation of Issue Dependencies
--------------------------------
Sometimes, while working on an issue, it becomes apparent that the
problem being worked on is actually a compound issue of various
smaller issues. Being able to take the current work and then begin
working on a separate issue is very helpful to separate out issues
into individual units of work instead of compounding them into a
single, large unit.
* Create a branch A (e.g. urllib has a bug).
* Edit some code.
* Create a new branch B that branch A depends on (e.g. the urllib
bug exposes a socket bug).
* Edit some code in branch B.
* Commit branch B.
* Edit some code in branch A.
* Commit branch A.
* Clean up.
svn
'''
To make up for svn's lack of cheap branching, it has a changelist
option to associate a file with a single changelist. This is not as
powerful as being able to associate at the commit level. There is
also no way to express dependencies between changelists.
::
cp -r trunk issue0000
cd issue0000
# Edit some code.
echo "The cake is a lie!" > README
svn changelist A README
# Edit some other code.
echo "I own Python!" > LICENSE
svn changelist B LICENSE
svn ci -m "Tell it how it is." --changelist B
# Edit changelist A some more.
svn ci -m "Speak the truth." --changelist A
cd ..
rm -rf issue0000
bzr
'''
Here's an approach that uses bzr shelf (now a standard part of bzr)
to squirrel away some changes temporarily while you take a detour to
fix the socket bugs.
::
bzr branch trunk bug-0000
cd bug-0000
# Edit some code. Dang, we need to fix the socket module.
bzr shelve --all
# Edit some code.
bzr commit -m "Socket module fixes"
# Detour over, now resume fixing urllib
bzr unshelve
# Edit some code
Another approach uses the loom plugin. Looms can
greatly simplify working on dependent branches because they
automatically take care of the stacking dependencies for you.
Imagine looms as a stack of dependent branches (called "threads" in
loom parlance), with easy ways to move up and down the stack of
threads, merge changes up the stack to descendant threads, create
diffs between threads, etc. Occasionally, you may need or want to
export your loom threads into separate branches, either for review
or commit. Higher threads incorporate all the changes in the lower
threads, automatically.
::
bzr branch trunk bug-0000
cd bug-0000
bzr loomify --base trunk
bzr create-thread fix-urllib
# Edit some code. Dang, we need to fix the socket module first.
bzr commit -m "Checkpointing my work so far"
bzr down-thread
bzr create-thread fix-socket
# Edit some code
bzr commit -m "Socket module fixes"
bzr up-thread
# Manually resolve conflicts if necessary