Skip to content
Open
Show file tree
Hide file tree
Changes from 11 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 27 additions & 26 deletions src/site/markdown/about-checksums.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,44 +19,45 @@ under the License.
-->

Maven Resolver uses checksums to verify the integrity of downloaded artifacts and
metadata. Checksums are usually laid out in repositories next to the file in question, with file
extension telling the checksum algorithm that produced the given checksum file content. Currently,
most Maven repositories contain SHA-1 and MD5 checksums by default (they are produced by Resolver by default).
metadata. Checksums are usually placed in repositories next to the file in question, with the file
extension indicating the checksum algorithm that produced the given file. Currently,
most Maven repositories contain SHA-1 and MD5 checksums as they are produced by Resolver by default).
Comment thread
elharo marked this conversation as resolved.
Outdated

Historically, Maven Resolver used `java.security.MessageDigest` to implement checksums. So to say, secure one-way
hashes provided by Java Cryptography Architecture were (mis)used to implement checksums for transport integrity
validation. There is no misunderstanding here, secure hashes MAY be used as checksums, as there is quite some
Historically, Maven Resolver used `java.security.MessageDigest` to implement checksums. Secure one-way
hashes provided by the Java Cryptography Architecture were (mis)used to implement checksums for transport integrity
validation. Secure hashes MAY be used as checksums, as there is quite some
overlap between checksums and hashes in general. But this simplicity comes at a price: cryptographically safe
algorithms require way more CPU cycles to calculate checksum, while all their purpose is just
integrity validation, nothing more. There is no security, trust or whatever else implied or expected from
them.
algorithms require way more CPU cycles to compute than a simple checksum. However, the purpose of a checksum is just
integrity validation, nothing more. There is no security or trust implied or expected from
them. Checksums do not protect against man-in-the-middle or supply chain attacks.

If you are interested in trust in your artifacts, it is signatures (for example
[GPG Signatures](https://maven.apache.org/plugins/maven-gpg-plugin/)) that you should look for.
To actually trust that artifacts have not been tampered with, you need signatures such
those provided by the
[Maven GPG Plugin](https://maven.apache.org/plugins/maven-gpg-plugin/)).
Comment thread
elharo marked this conversation as resolved.
Outdated

Hence, the usual argument that "XXX algorithm is unsafe, deprecated, not secure anymore" does not stand in use case
of Maven Resolver: there is nothing secure being involved with checksums. Moreover, this is true not only for SHA-1
algorithm, but even for its "elder brother" MD5. Both algorithms are still widely used today as "transport integrity
validation" or "error detection" (aka "bit-rot detection").
Hence, the usual argument that "XXX algorithm is unsafe, deprecated, not secure anymore" does not apply in the case
of Maven Resolver. Moreover, this is true not only for SHA-1
algorithm, but even for its "elder brother" MD5. A checksum is not intended to be secure. Both algorithms are still widely used today as "transport integrity
validation" or "error detection" (a.k.a. "bit-rot detection").

## Checksum Algorithms SPI

From a technical perspective, the above written facts infer following consequences: as checksum algorithms are exposed
to the user, so one can set them via configuration, users are not prevented to ask for SHA-256 or even SHA-512, even if
these algorithms are not part of standard Maven process. Moreover, nothing prevent users (integrating
Maven Resolver) registering with Java an alternate Java Cryptography Provider and use even broader (or exotic) set
of message digests for checksums. While this is not wrong or even mistake in any case, we do consider this as
wrong use case. The notion of transport validation and secure hashes are being constantly mixed up due historical
From a technical perspective, the above facts imply the following consequences: because checksum algorithms are exposed
to the user, one can set them via configuration, and thus users are not prevented from asking for SHA-256 or even SHA-512, even if
these algorithms are not part of standard Maven process. Moreover, nothing prevents users (integrating
Maven Resolver) registering an alternate Java Cryptography Provider and using even broader (or exotic)
message digest algorithms for checksums. While this is not wrong, we do consider this as a
bad use case. The notion of transport validation and secure hashes are being constantly mixed up due to historical
reasons explained above.

Hence, Maven Resolver team decided to make supported set of checksums more controlled. Instead of directly exposing
`MessageDigest` algorithms, we introduced an SPI around checksums. This not only prevents wrong use cases (not
exposing all supported algorithms of `MessageDigest` to users), but also makes possible to introduce real checksum
Hence, the Maven Resolver team decided to make the supported set of checksum algorithms more controlled. Instead of directly exposing
`MessageDigest` algorithms, we introduced an SPI around checksums. This not only prevents incorrect use cases by not
exposing all supported algorithms of `MessageDigest` to users, but also makes it possible to introduce real checksum
algorithms. Finally, the set of supported checksum algorithms remains extensible: if some required algorithm is
not provided by Resolver, it can be easily added by creating a factory component for it.
not provided by Resolver, it can easily be added by creating a factory component for it.

We are aware that users started using "better SHA" algorithms, and we do not want to break them. Nothing for them
changes (configuration and everything basically remains the same). But, we do want to prevent any possible further
changes (configuration and everything basically remains the same). But we do want to prevent any possible further
proliferation of non-standard checksums.

## Implemented Checksum Algorithms
Expand Down
22 changes: 10 additions & 12 deletions src/site/markdown/api-compatibility.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,11 +70,9 @@ changes using version number. We use "major.minor.patch" versioning on resolver
semantics:

* On major version change, one should NOT expect any backward compatibility.
* On minor version change, we TRY to keep backward compatibility for those "exposed" 3 modules:
API, SPI and Util. Still, there are examples when we failed to do so, usually driven by new
features.
* On minor version change, we ENSURE backward compatibility for those "exposed" 3 modules: API,
SPI and Util.
SPI and Util. Still, there are examples when we failed to do so, usually driven by new
features.
Comment on lines 73 to +75

In any of three version changes above, in areas where we do not offer guarantees, everything
can happen.
Expand All @@ -88,26 +86,26 @@ above.

## Inside of Maven

Historically, Maven 3.1 (as Maven 3.0 used resolver from different package) provided API, SPI
and Impl from its own embedded resolver, while Util, Connector, if some plugin or extension
depended on those, was resolved. This caused that a plugin may work with different versions
of API, SPI, Impl or Connector. Given Resolver had API "frozen" for too long time, this was essentially
Historically, Maven 3.1 provided API, SPI
and Impl from its own embedded resolver, while Util and Connector, if some plugin or extension
depended on them, were resolved separately. This meant that a plugin could work with different versions
of API, SPI, Impl or Connector. Because the Resolver API was "frozen" for too long a time, this was essentially
not a problem, but still weird.

This changes in Maven 3.9+: Maven starting with version 3.9.0 will provide API, SPI, Impl
This changes in Maven 3.9+: Maven starting with version 3.9.0 will provide API, SPI, Impl,
**and Util and Connector**. Reason for this change is that Impl and Connector bundled in Maven
implements things from both, API and SPI, and there was a binary incompatible change between
implement things from both API and SPI, and there was a binary incompatible change between
Resolver 1.8.0 and previous versions.

Most Resolver users should not be affected by this change.

The binary incompatible change happened in SPI class `RepositoryLayout` as part of work done for
The binary incompatible change happened in the SPI class `RepositoryLayout` as part of work done for
[MRESOLVER-230](https://issues.apache.org/jira/browse/MRESOLVER-230), and affects both, Connector
and Impl.

## Backward Compatibility Checks

To ensure backward compatibility, starting from 1.9.0 version Maven Resolver uses
To ensure backward compatibility, starting from 1.9.0 Maven Resolver uses
[JApiCmp](https://siom79.github.io/japicmp/MavenPlugin.html),
with two executions (for source and binary level checks). The plugin is enabled on 3 modules of
Resolver mentioned at page top: API, SPI and Util. For "baseline" we use version 1.8.0.
43 changes: 22 additions & 21 deletions src/site/markdown/common-misconceptions.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,11 @@ under the License.

Due to smooth transitions from Maven2 into Maven3 (and soon
Maven4), and the fact that Maven2 plugins kept working with Maven3, maybe
even without change, there were some misconceptions crept in
even without change, some misconceptions crept in
as well. Despite the marvel of "compatibility", Maven3 resolution
differs quite much from Maven2, and the sole reason is actual improvement
in area of resolution, it became much more precise (and, due
that lost some "bad" habits present in Maven2). Here, we will try to
differs considerably from Maven2, and the sole reason is actual improvement
in area of resolution. It became much more precise (and, due
to that, lost some "bad" habits present in Maven2). Here, we will try to
enumerate some of the most common misconceptions.

## Misconception No1: How Resolver Works
Expand All @@ -36,7 +36,7 @@ dependencies. Resolver, to achieve this, internally (but these are
exposed via API as distinguished API calls as well) performs 3 steps:
"collect", "transform" and "resolve".

The "collect" step is first, where it builds the "dirty tree" (dirty graph)
The "collect" step is first, where it builds the "dirty tree" (or dirty graph)
of artifacts. It is important to remark, that in "collect" step, while
the graph is being built, Maven uses only POMs. Hence, if collecting an
Artifact that was never downloaded to your local repository, it will
Expand Down Expand Up @@ -81,8 +81,8 @@ a showcase how Resolver works.
## Misconception No2: "Test graph" Is Superset Of "Runtime graph"

**Wrong**. As can be seen from above, for runtime graph we leave out "test" scoped
dependencies. It was true in Maven2, where test graph really was superset of runtime,
this does not stand anymore in Maven3. And this have interesting consequences. Let me show an example:
dependencies. It was true in Maven2, where test graph really was a superset of runtime,
but this does not stand anymore in Maven3. And this has interesting consequences. Let me show an example:

(Note: very same scenario, as explained below for Guice+Guava would work for Jackson Databind+Core, etc.)

Expand All @@ -98,9 +98,9 @@ Assume your project is using Google Guice, so you have declared it as a dependen

All fine and dandy. At the same time, you want to avoid any use of Guava. We all know Guava is a direct dependency
of Guice. This is fine, since as we know, the best practice is to declare all dependencies your code compiles
against. By not having Guava here, analyse tools will report if code touches Guava as "undeclared dependency".
against. By not having Guava here, analysis tools will report if code touches Guava as an "undeclared dependency".

But let's go one step further: turns out, to set up your unit tests, you **do need** Guava. So what now? Nothing, just
But let's go one step further: to set up your unit tests, you **do need** Guava. So what now? Nothing, just
add it as a test dependency, so your POM looks like this:

```
Expand Down Expand Up @@ -136,17 +136,18 @@ The `dependency:tree` plugin for this project outputs this verbose tree:
[INFO] \- com.google.j2objc:j2objc-annotations:jar:1.3:test
```

And is right, this IS the "test graph" **of the project** and contains a conflict as noted by "omitted for duplicate"
This IS the "test graph" **of the project** and contains a conflict as noted by "omitted for duplicate"
and "scope not updated to compile" remarks next to Guava nodes.

So this setup results that:
* when you compile, it ensures Guava is NOT on compile classpath, so you cannot even touch it (by mistake)
* when test-compile and test-execute runs, Guava will be present on classpath, as expected

So far good, but what happens when this library is consumed downstream by someone? When it becomes used as a library?
* when you compile, Guava is NOT on compile classpath, so you cannot even touch it (by mistake)
* when test-compile and test-execute run, Guava will be present on classpath, as expected

So far so good, but what happens when this library is consumed downstream by someone? When it becomes used as a library?
Nothing, all works as expected!

When a downstream dependency declares dependency on this project, the downstream project will get this graph (from
When a downstream dependency declares a dependency on this project, the downstream project will get this graph (from
the node that is your library):

```
Expand All @@ -166,29 +167,29 @@ the node that is your library):
[INFO] \- com.google.j2objc:j2objc-annotations:jar:1.3:compile
```

So what happens here? First, revisit "How Resolver Works", there you will see that for "runtime graph" of the
So what happens here? First, revisit "How Resolver Works". There you will see that for "runtime graph" of the
dependency the "test" and "provided" scopes of the dependency artifact **are not even considered**. They are simply
omitted. Not skipped, but completely omitted, like they do not even exists. Hence, in the graph there is
omitted. Not skipped, but completely omitted, like they do not even exist. Hence, in the graph there is
**no conflict happening** (as "test" Guava is completely omitted during "collect" step). Hence, everything
goes as expected.

### Important Consequences

One, maybe not so obvious consequence can be explained with use of `maven-assembly-plugin`. Let assume you want to
One, maybe not so obvious consequence can be explained with use of `maven-assembly-plugin`. Assume you want to
assemble your module "runtime" dependencies.

If you do it from "within" of the project, for example in package phase, your packaging will be incomplete:
If you do it from "within" the project, for example in the package phase, your packaging will be incomplete.
Guava will be missing! But if you do it from "outside" of the project (i.e. subsequent module of the build, or
downstream dependency), the assembly will contain Guava as well.

This is a [Maven Assembly plugin bug](https://issues.apache.org/jira/browse/MASSEMBLY-1008), somewhat explained
in [MRESOLVER-391](https://issues.apache.org/jira/browse/MRESOLVER-391). In short, Maven Assembly plugin considers
in [MRESOLVER-391](https://issues.apache.org/jira/browse/MRESOLVER-391). In short, the Maven Assembly plugin considers
"project test graph", and then "cherry-picks runtime scoped nodes" from it, which, as we can see in this case,
is wrong. You need to build different graphs for "runtime" and "test" classpath, unlike as it was true in Maven2.
is wrong. You need to build different graphs for "runtime" and "test" classpath.
For Assembly plugin, the problem is that as Mojo, it requests "test graph", then it reads configuration
(assembly descriptor, and this is the point where it learns about required scopes), and then it "filters"
the resolved "test graph" for runtime scopes. And it is wrong, as Guava is in test scope. Instead, the plugin
should read the configuration first, and ask Resolver for "runtime graph" and filter that. In turn, this problem
does not stand with `maven-war-plugin`, as the "war" Mojo asks resolution of "compile+runtime" scope. Of course,
WAR use case is much simpler than Assembly use case is, as former always packages same scope, while Assembly receives
the WAR use case is much simpler than the Assembly use case is, as the former always packages the same scope, while Assembly receives
a complex configuration and exposes much more complex "modus operandi".
7 changes: 4 additions & 3 deletions src/site/markdown/creating-a-repository-system-session.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,17 +30,18 @@ represented by an instance of
setup can be done like this:

```java
import org.eclipse.aether.DefaultRepositorySystemSession;
import org.eclipse.aether.supplier.RepositorySystemSupplier;

...
private static RepositorySystemSession newSession( RepositorySystem system )
{
RepositorySystemSession.SessionBuilder sessionBuilder = SessionBuilderSupplier.get();
DefaultRepositorySystemSession session = new DefaultRepositorySystemSession();

LocalRepository localRepo = new LocalRepository( "target/local-repo" );
sessionBuilder.setLocalRepositoryManager( system.newLocalRepositoryManager( session, localRepo ) );
session.setLocalRepositoryManager( system.newLocalRepositoryManager( session, localRepo ) );

return session.build();
return session;
}
```
Comment on lines +38 to 45

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fix correctly replaces the undefined session variable and SessionBuilderSupplier, but system.newLocalRepositoryManager(sessionBuilder, localRepo) will not compile: newLocalRepositoryManager expects a RepositorySystemSession as its first argument, and SessionBuilder does not extend RepositorySystemSession.

The SessionBuilder API provides withLocalRepositories(LocalRepository...) for this purpose, which would make the example both simpler and correct:

Suggested change
RepositorySystemSession.SessionBuilder sessionBuilder = system.createSessionBuilder();
LocalRepository localRepo = new LocalRepository( "target/local-repo" );
sessionBuilder.setLocalRepositoryManager( system.newLocalRepositoryManager( session, localRepo ) );
sessionBuilder.setLocalRepositoryManager( system.newLocalRepositoryManager( sessionBuilder, localRepo ) );
return session.build();
return sessionBuilder.build();
}
```
RepositorySystemSession.SessionBuilder sessionBuilder = system.createSessionBuilder();
LocalRepository localRepo = new LocalRepository( "target/local-repo" );
sessionBuilder.withLocalRepositories( localRepo );
return sessionBuilder.build();


Expand Down
4 changes: 2 additions & 2 deletions src/site/markdown/deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,9 @@ Deploying artifacts and related metadata to a (remote) repository can be achieve

The most prominent consumer of this API is probably [maven-deploy-plugin](https://maven.apache.org/plugins/maven-deploy-plugin/).

# Repository Connector
## Repository Connector

The default repository connector implementation at [`BasicRepositoryConnectorFactory`](https://github.com/apache/maven-resolver/blob/master/maven-resolver-connector-basic/src/main/java/org/eclipse/aether/connector/basic/BasicRepositoryConnector.java) uses a `RepositoryLayout` to calculate the URL and a `Transporter` to achieve the actual upload/write of artifacts/metadata.
The default repository connector implementation at [`BasicRepositoryConnector`](https://github.com/apache/maven-resolver/blob/master/maven-resolver-connector-basic/src/main/java/org/eclipse/aether/connector/basic/BasicRepositoryConnector.java) uses a `RepositoryLayout` to calculate the URL and a `Transporter` to achieve the actual upload/write of artifacts/metadata.

## Repository Layout

Expand Down
Loading
Loading