Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from https://github.com/npm/npm/issues/5239 below:

"scoped" packages · Issue #5239 · npm/npm · GitHub

The biggest request from enterprise users is a way to manage private modules with the same level of simplicity and ease as open source modules in the public npm registry. (A close second is group/role based management, but let's leave that for a subsequent discussion.)

Since the registry is (mostly) a Couch App, and we expose a public CouchDB interface to it, you can replicate the metadata into a database on your network, and optionally also use the npm-fullfat-registry module to fetch binaries into your copy of the db as attachments.

However, this is suboptimal in the following ways:

1. You end up replicating hundreds of GB of stuff that you don't need.
2. CouchDB with half a million binary attachments is not an easily maintained beast.

PayPal wrote kappa to deal with this, which doesn't require using a CouchDB full of attachments. There are several other similar proxy implementations that store the metadata and binaries to disk, but this is the one that I hear of the most often. This is a bit better than managing a Couch App, and has pluggable interfaces for caching, whitelists, etc.

Another option is to use private git repositories, either managed in-house or on GitHub. The npm client has some snazzy shorthands for GitHub repos, so you can do "foo":"org/node-foo" in your package.json dependencies, or npm install org/node-foo on the cli. However, there is a bit of an impedance mismatch and occasional friction in using git repos as npm dependencies. It's ok in a pinch, or if you have really well-established practices, but it's generally always going to be a bit of a mismatch, simply because git and npm make different assumptions about the world.

If you use basically any private system, you'll end up with name conflicts. Let's say our company creates a module called marked, to track places that we've marked in our location tracker app. Then we decide to use some public module that depends on @chjj's marked module, and it ends up installing our location tracker doohickey as a dep, because the registry is set to npm.my-company.com.

We can get around this by implementing some rather convoluted logic to manage multiple registries, and keeping track of where on the list of registries we started in the lookup process, and always go down the list rather than up.

However, this means that we can't do things like fork the markdown parser package named marked to fix a bug, and have our dependency lookups find that instead. There is no way to solve both problems, because they contradict one another.

What's worse, the logic for knowing how to resolve conflicts gets really confusing to reason about, and when you're looking at code, it's not clear what require("marked") refers to.

So, we have to be very careful to not use names that are publicly used, or squat on public names, which disrupts the OSS community flow, and so is generally bad behavior.

It'd be nice if we could scope modules to a specific user or organization. If such scoped modules were able to be made private, and if they could be unambiguously recognized by the client, then it would know that it might have to do a "give me access to this private thing" authorization dance in those cases.

Some constraints:

1. There must be a simple and unambiguous way to say "install mycompany's marked module".
2. It must not conflict with any currently allowed names. (Ie, it should be invalid in the current paradigm. Not hard to do, since names are very restrictive.)
3. It should not conflict with the user/repo shorthand used for GitHub. (Oops. That would've been a really nice way to go. Oh well, backwards compatibility. It sucks, but not as much as it would suck to break users.)
4. If you do npm install <scoped module> then you should be able to do require("<scoped module>") in your JavaScript, and view the data at https://registry.npmjs.org/<scoped module> in a web browser.
5. Since (3) implies we have some kind of magic character or syntax, (4) means that any such magic character/prefix/syntax must not use characters that are magical in URLs, file systems, or shells. (For example, npm install ~org/foo or npm install $org/foo would be fine in the fs and url, but the shell will expand ~org to the home of user org and $org to the value of the org var. + and # and % all have similar problems in other places.)

I've discussed this with the people here at npm, Inc., and with a few of the most prolific npm contributors and publishers, and users at several large companies. The scheme that seems to be the most intuitive, which also fits within the rather narrow constraints is @org/pkgname.

Each @org will map to a specific registry in the config (or the default --registry config, which defaults to registry.npmjs.org, if not specified).

We can then also scope auth info to each registry, and voila, we have multiple registry support that works, doesn't conflict with anything that already exists, doesn't result in fetching the right thing from the wrong place.

@org/pkg modules will be dropped onto the filesystem at node_modules/@org/foo, so require("@org/pkg") will work.

And since that's the actual package name, you'll do "dependencies":{"@org/pkg":"1.x"} in your package.json file.

As a way of implementation details, the @org/pkg thing will map to npm://{registry}/@org/pkg, where {registry} is the registry that is assigned to the @org or the default --registry config. npm:// urls will be fetched via https, and always send authentication information, and treat 404's as a potential "maybe you don't have permission" response.

This does mean that the registry will probably not be strictly just a Couch App for much longer. In practice, it already isn't, but since CouchDB chokes on doc names that have a / in them means that we'll have to %2F it, and that's just a bit confusing. Maybe we could use - instead of /, but the niceness of / as a filesystem separator is really awesome.

We'll still keep shipping an OSS reference implementation, but it probably won't do private modules, since that usually is going to require some kind of custom logic. Basically, npm-registry-couchapp will be just a part of the registry implementation, instead of effectively the entire thing. In addition, there'll be a minimal "front door" bit that proxies to the Couch App, manages roles, restricts GETs, etc., and that's where you'll find hooks for auth magic. That also means we can avoid putting the attachments into CouchDB in the first place, since all we do is then immediately take them out of CouchDB, without breaking old clients, and opens the door to doing other creative stuff, like more efficient views, relational reporting stuff, and putting download counts and other analytics right into the data itself. Handwavey vaporware, etc.

Would you use this? What would you like it to do? Anything about it that you think would be weird or confusing? Is there an approach you'd like better that meets all the constraints?

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4