The main DAOS repository is hosted on hpdd.intel.com and a mirror is maintained on GitHub. Gerrit is primarily a code review tool, but can also be used to track the status of patch as it is build tested, and finally merged to a branch. Developers submit patches to Gerrit using git, and there is a rich browser interface for performing code reviews. Reviewers can also pull the change directly to their own repository and review the change locally.
To upload changes to Gerrit, you must first register and add an ssh key to your account. For details on setting up your Gerrit account and sending patches, please see the Gerrit documentation.
Gerrit can host many git repositories, and of course each of those repositories can contain many branches.
The IOR DAOS driver, the MPI-IO DAOS driver and the go binding for the DAOS API are hosted on GitHub. Please find below a summary of all the repositories available today:
next DAOS release
Open for new features and bug fixes
Mirror of daos/daos_m
Mirror of upstream IOR
Include DAOS IOR driver
|Open for new features and bug fixes|
|daos-stack/go-daos||branch: master||Go binding for the DAOS API|
Contributions to the GitHub repositories should submitted through GitHub pull requests.
Managing Changes in Git
Whole books could be written about this topic, and there plenty of online tutorials on the web that explain this in more detail and suggest other methods of managing changes. However, this distilled version is (hopefully) enough to get started.
To make an initial checkout of the master DAOS Git repository:
This will create the
master branch in your repository, which should never be modified. Typically in git, one creates a new local branch for each change being made. The branch is created based on one of the local, pristine branches follow the main development branches, usually the
master. A new local branch is usually create using checkout:
This creates a new branch based on
my-brilliant-idea, and also makes it the current branch.
In order ensure that the commit description contains the correct name and email address for you, it is possible to specify this directly to Git creating or modifying the
$HOME/.gitconfig file in your home directory (preferred), or in the
.git/config file in the local repository. This only needs to be done one time, or once per checkout, if done in the local repository.
commit commands in any repository will use this name/email regardless of which user account is used to do the modifications, if using
$HOME/.gitconfig file. If you want to specify a different name or email for a specific repository, it is possible to add this information to the
.git/config file in that specific repository. See commit comment details below. The optional
[remote "review"] block adds an alias for the remote Gerrit repository to simplify pulling updates and pushing patches, and should only be added in the local
Once you've made the change, and you want to save the code for testing and reviewed by others, you
commit the change. Before you commit in Git, you need to identify which new or changed file(s) you want to commit using
git add <filename> for a specific filename, or
git add -u for all modified files. If you haven't added any new files, and you want to commit all the files that you have changed (which is the usual case), then you can use the
-a option to commit:
This will put you into an editor to update your commit comment, and when you save it will commit the change locally. The
-v option appends the diff to the edit buffer so you can review what is about to be committed. This is very useful to verify that you are committing what you want to commit, and not changes or files that are unrelated to the current change. The
-s option adds a
Signed-off-by: line to the commit message, which is required for all patches submitted to DAOS.
It is best to commit relatively frequently to keep your changes limited to a single fix. If you notice other, unrelated, issues that need to be fixed, then it's best to put them in a separate commit on a separate branch, so they can be submitted to Gerrit independently for review, testing, and landing. The
stash command is handy for this:
Formatting Git Commit Comments
Having good commit comments helps everyone that is working on the code. See Commit Comments for a detailed description of the commit comment style. If your commit does not include the
Change-Id: line as described below (which should be added automatically by the
commit-msg hook, see above), the patch will be automatically rejected at submission time.
Sample Commit message:
Adding the Change-Id field
Gerrit requires all changes to have the
Change-Id: field, or it will be rejected. It should be added automatically by the
.git/hooks/commit-msg script. See Commit Comments for a detailed description of the
Submitting Patches for Review, Testing, and Landing
A change request is created by pushing a patch to a special branch on the Gerrit repository. To create an change request for a patch in your local branch the local commit (or series of commits) needs to be pushed to the Gerrit review repository.
It is possible to push a series of dependent commits from a local branch in this same way, and each one will create a separate change in Gerrit. Patch series should be used to split complex code changes that are implementing a single larger feature into chunks that are easier to understand, review, test, and land. Changes in a patch series will be dependent on each other, so if one patch needs to be refreshed after a review or test failure it will cause all of the later patches to be refreshed also. That will clear all of the review and test results and restart testing on all of the patches.
Patch series should not be used for code that was written and then had a series of bug fixes applied to it after local testing. The later (local) bug fixes should be squashed into the original change before submission. Note that each patch in a series must build and test properly before it can be landed, so compile warnings or test failures in the middle of a patch series are not allowed.
If you have a series of independent commits to be reviewed, each one should be in a separate local branch and pushed separately to Gerrit. This allows the patches to be reviewed, tested, and landed independently, and will avoid the overhead and delay of repeatedly reviewing, building, and testing patches that are only refreshed because of an earlier (unrelated) patch in a series being modified.
master branch of the daos
For convenience, you can add this to your
Creating a review request for a change against master (assuming the remote alias has been added to ssh config):
Add the Gerrit server as a remote to your git repository:
Automatically Building and Testing a Patch
When the patch has been pushed to the Gerrit
daos/daos_m repository, it will print a URL for that change set that should be added to the Jira ticket for tracking. This should now happen automatically.
All patches submitted to Gerrit will be built automatically by Jenkins for a number of different distro and kernel versions. Currently for the master branch this includes RHEL6 i686/x86_64 client/server, RHEL7 x86_64 client, and SLES11 x86_64 client/server. The success/failure of these builds will be posted to the change in Gerrit as a link to the Jenkins build artifacts.
After a successful build on all of the configurations, one or more of the configurations will be regression tested automatically, and a link to the test results in Maloo will be posted to Gerrit.
Automated regression testing for each patch should normally pass. In some cases it might fail for a variety of reasons, such as a bug in the patch, an intermittent test failure that is present in the existing code, problems with the testing system, etc. In all failure cases, the Maloo test failure(s) should be investigated as to the root cause. Once the test failure is identified in Maloo, the failed test result should either be linked (
Associate bug) to an existing DAOS ticket in Jira, or if necessary a new Jira ticket with details of the failure should be filed (
Raise bug), and a comment added to Gerrit with the JIRA DAOS ticket number. At this point the test could be resubmitted, if there are no other problems with the patch, or it can be refreshed if there are review comments that need to be addressed (per below).
Requesting Patch Review
In order to actually get a patch landed, you need to request at least two reviews in Gerrit for the patch. This is done by visiting the Gerrit web page for the change (at the URL returned when the commit was pushed, or it can be found from your home page after logging in). Enter the name or email address for the reviewer and click the Add... button in the top right corner.
In conjunction with the two review requests, the patch has to successfully build on all supported architecture/distro combinations, and pass the automatic testing (as reported by Maloo). Once the patch has passed two reviews (+1 or +2 Review from reviewers that are not the original developer), built correctly (+1 Verified from Jenkins), and has passed autotest (+1 Verified from Maloo) the patch will automatically appear in the Gerrit Gatekeeper's list of patches to land.
It is the responsibility of the patch submitter to request the reviewers to look at the patch or to request patch landing, if this is not happening in a timely manner, by adding a comment into the patch.
Updating Patches After Review
Gerrit makes it easy to update a patch after review, and doing this allows reviewers to see differences between the patches so they only need to review the changes between the patches instead of having to review the entire patch again. Also, the original inline comments are maintained and moved to the new patches. The key to keeping updated review requests linked to the original patch is the
Change-Id: field in the commit comment - this is what Gerrit uses to find the original patch to update.
A critical thing to point out is that you must submit a new version of the entire patch - not just an update to the patch.
The easiest way to update the most recent commit (which is often the one you want to update), is to use "
git commit --amend -a". This will
"add" any modifications in the current repository and merge them into the last commit. If there are no changes, or "
-a" is not used, this will just allow you to edit the most recent commit message. This is useful if you don't have a
Change-Id: line in your commit message (because you didn't install the
commit-msg hook, see Commit Comments above), but one was added automatically by Gerrit. You can copy the
Change-Id:line from the Gerrit web page for that change and paste it into the amended commit message. Typically, the updated patch should also be rebased to the tip of the current branch before pushing it again, to ensure that it does not conflict with other changes that may have been landed since the patch was first developed.
Another way of modifying a series of existing changes is to use "
git rebase -i <parent-branch>". Interactive rebase will display a list of patches in an editor, and you can reorder the patches, "
edit" the patch, "
reword" the commit comments, and "
squash" two patches into a single patch. See the help text in the commit message editor window for more information.
Once the change has been updated, push your branch to Gerrit again, and the original request will be updated with the new version of the patch.
(Re-)Basing One Change on Another Change
It is possible to base a new change, or rebase an existing change, on an uncommitted patch in Gerrit. This might be useful if both changes are impacting the same code, and one change is clearly dependent on another for some reason, or if they will cause conflicts when merged and a decision is made to land them in a specific order. It is desirable to break a patch into multiple functionally-separate commits. If the patches are truly independent (i.e. no overlapping changes to the same files, no dependent functions, etc), then those patches should be submitted in separate branches, and this section does not apply.
If both patches are being developed by the same person, the easiest way to have a series of dependent changes is to commit them into separate patches order on the same branch. Then, when a change is made to any patch on that branch, all of the dependent changes will also be resubmitted to Gerrit so that they will be ready to land when the earlier patches are merged.
If updates or fixes need to be made to one of the patches, these updates should be merged into the original commit where the code was added, rather than being an additional patch at the end of the series. This can be done by running
"git rebase -i <parent-branch>", then marking a particular patch for
edit. That will cause the earlier patches in the series up to the to be applied, then stop for interactive editing/testing of that patch. Once the patch has been updated, run
git add -u to include the new updates into this patch, and
git rebase --continue to merge the updates into the existing patch and continue the rebase process.
In the case where changes need to be based on a patch from another developer, it is possible to check out the desired patch from Gerrit using one of the supplied Git URLs in the
Download section under the patchset. Select the
Anonymous HTTP or
SSH options, then copy the URL and paste it at the command prompt of a previously checked-out DAOS tree. For example to fetch patchset 42 from https://review.hpdd.intel.com/31188/, create a branch for it, and then create a new branch for your local changes based on that change, use:
Any changes in the
b_31188_my_changes will be based on top of those of
b_1264_my_changes is pushed to Gerrit, it will have a dependency on change 31188, so it will not be able to land until change 31188 itself is landed. It is typically not desirable rebase the
b_31188 branch to a new version of
master or modify it, or it will cause the other developer's patch to be updated in Gerrit, and lose its existing review and test results. However, in some cases this is desirable, and Gerrit allows a user other than the original author to push an updated version of the patch (using the same
Change-Id: label) in case there is a problem with the original patch (e.g. no longer applies, has a bug, etc).
If you already have the changes in a local branch (e.g.
b_my_changes) and want to rebase that branch on top of another uncommitted patch from Gerrit, the process is similar:
That will rebase the patch(es) in the current branch onto the other patch (or branch) checked out from Gerrit.
Pushing a Git Tag
This is normally done by the branch maintainer, but is recorded here for future reference.
Switch to the branch you want to create a branch in.
Create the tag with:
then push it upstream with
Note, this requires certain privileges on the server. If
review is a secondary server for your repository, use: