Hey Frederic, > But then, this is not an import problem. > If you use Numeric, you call Numeric. If you call something other than Numeric, > just give a different name, and all the confusion will go away. This is certainly an option, but not a good one in my opinion. The main issue is we want to force a specific version of Numeric for SciPy while allowing people to keep their old standard version of Numeric available for their production code. Single level packages provide a handy way of doing this. Multi-level packages (like SciPy) do not. I guess I just don't see why making a package multi-level should inherently make it harder to do things. > If you're worried that you've already encoded the Numeric name 50 times into > 300 files; run a python script over these 300 files; this will do the renaming > of the 15.000 occurences of the Numeric name. Sure, but this is inconvenient, and something I think should be handled by the packaging facility, not by running a renaming script. > > > That makes SciPy > > self-contained and allows people to try it out without worrying that it > > might break their current installation. There are other solutions to this > > problem, but Prabhu's fix is by far the easiest and most robust. > > And then, in maintenance/integration phase, sometimes 'Numeric' will call > Numeric, some other times it will your package ? In the integration phase, users would need to change the "from Numeric import *" to "from scipy import *" (or search and replace Numeric with scipy) in their code as Numeric is completely subsumed into scipy. So, when not using SciPy, their legacy code can continue using an old version of Numeric. When switching to SciPy, the make the replacement. As I said, its mainly a version issue (with a few minor changes). > > What if somebody, for some reason I know nothing of (e.g. probably some > integration) wants to call Numeric and your Numeric package in the same module ? > Wish them tough luck to sort out this poisoned gift.... > > > Prabhu's import also has some other nice benefits. Some of the sub-packages > > in SciPy are useful outside of SciPy. Also sometimes it is easier to > > develop a packages outside of the SciPy framework. It would be nice to be > > able to develop a module or package 'foo' outside of SciPy and then move it > > into SciPy at a later date. However, every SciPy sub-package that referred > > to foo prior to its inclusion in SciPy now has to be updated from 'import > > foo' to 'import scipy.foo'. These kind of issues make it very painful and > > time consuming to rearrange package structures or move modules and > > sub-packages in and out of the package. > > There are basic python scripts which do this painlessly. If you're really > working on a large project, there's a project architect which normally would > take care of such things, and for whom this should not be a too much of a > problem. Hmmm. I guess the "project architect" in this case is jointly held by Travis Oliphant and yours truely. Neither of us are packaging guru's, but do have a fair amount of experience with Python. We worked quite a while on (and are still working on) packaging issues. Incidently, I have know idea what Travis O.'s opinion is on this specific topic. > > > > Simplifying this will improves > > package development. > > > > > I'm personnally against anything that enlarges the search path uselessly; > > > > Hopefully I've explained why it is useful for complex packages. > > Python helps in many areas, but expecting it to palliate for the package design > and architecture flaws that inexorably surface anytimes something non-trivial is > developped, might be somehow at the edge. Python has not yet replaced the need > for relevant software architects. Them thars fightin' words. ; ) I'm biased, but don't thinking scipy's architecture is flawed. It is simply a *very* large package of integrated sub-packages that also relies heavily on a 3rd evolving group of modules (Numeric). As such, it reveals the difficult issues that arise when trying to build large packages of integrated sub-packages that rely on a 3rd evolving group of modules... > > > > > > because the obvious reason of increased name space collision, increased > > > run-time overhead etc... > > > > I'm missing something here because I don't understand why this increases > > name space collision. If the objection is to the fact that SciPy can have a > > version of Numeric in it that masks a Numeric installed in site-packages, I > > guess I consider this a feature, not a bug. > > Actually, it is normally worse than a bug: it is a source of bug tomorrow in > your application - of all the bugs you'll have when your programmer will be > confusing the two Numeric packages, as well as all the mainteance and > integration problems you'll have down the line -. I disagree and don't think that is true in this (and many other) situations. People who want to use SciPy will migrate completely to it since it includes Numeric. What the sub-package option offers is a way to test SciPy and optionally use it while keeping their standard Numeric around for their production code. > > But by then, hopefully for you, you'll be somewhere else... The sad reality of > most projects :(( > > > Afterall, this is already the > > behavior for single level packages, extending it to multi-level packages > > seems natural. If this isn't your objection, please explain. > > > > The current runtime overhead isn't so bad. Prabhu sent me a few numbers on > > the SciPy import (which contains maybe 10-15 nested packages). I attached > > them below -- the overhead is less than 10%. It should be negligible for > > standard modules as only packages are really affected (right Prabhu?). > > And that's how, when you cumulate of the overheads for all new features, you get > potenially +100-200% overhead on the new releases. > Albeit all the efforts of the Python team, Python 2.0 is up to 70% slower than > python 1.5.2; Python 2.1.1 is up to 30% slower than python 2.0, and so on... > So, +10% on only such a minor features is anything but negligible :((( The computational cost of additional functionality is always a question of what portion of a program is impacted. If we were talking about 10% hit on looping structures or dictionary lookups or local variable lookups, then yes it needs extreme scrutiny. Adding 10% to a rare event is not worthy of note. I expect (and see) 0% overhead for importing standard modules (by far the most common case). Adding 10% overhead to importing a very large package with 10-15 nested sub-packages is just not a big deal. The 350% cost I saw (noted in a response to Gordon) is a *huge* deal and would need to be solved (moving to C would help) before this became standard. eric ----- Original Message ----- From: "Frederic Giacometti" <frederic.giacometti@arakne.com> To: "eric" <ej@ee.duke.edu> Cc: <import-sig@python.org>; <prabhu@cyberwaveindia.com>; <python-list@python.org>; <python-dev@python.org> Sent: Saturday, November 10, 2001 4:43 PM Subject: Re: Proposal for a modified import mechanism. > > > eric wrote: > > > I have to agree with Prabhu on this one. The current behavior of import, > > while fine for standard modules and even simple packages with a single > > level, is sub-optimal for packages that contain sub-packages. The proposed > > behavior solves the problem. > > > > Handling the packaging issues in SciPy was difficult, and even resulted in a > > (not always popular) decision to build and overwrite the Numeric package on > > machines that install SciPy. Prabhu's import doesn't resolve all the issues > > (I think packages may just be difficult...), but it would have solved this > > one. The proposed import allows us to put our own version of Numeric in the > > top SciPy directory. Then all SciPy sub-packages would grab this one > > instead of an existing site-packages/Numeric. > > But then, this is not an import problem. > If you use Numeric, you call Numeric. If you call something other than Numeric, > just give a different name, and all the confusion will go away. > If you're worried that you've already encoded the Numeric name 50 times into > 300 files; run a python script over these 300 files; this will do the renaming > of the 15.000 occurences of the Numeric name. > > > That makes SciPy > > self-contained and allows people to try it out without worrying that it > > might break their current installation. There are other solutions to this > > problem, but Prabhu's fix is by far the easiest and most robust. > > And then, in maintenance/integration phase, sometimes 'Numeric' will call > Numeric, some other times it will your package ? > > What if somebody, for some reason I know nothing of (e.g. probably some > integration) wants to call Numeric and your Numeric package in the same module ? > Wish them tough luck to sort out this poisoned gift.... > > > Prabhu's import also has some other nice benefits. Some of the sub-packages > > in SciPy are useful outside of SciPy. Also sometimes it is easier to > > develop a packages outside of the SciPy framework. It would be nice to be > > able to develop a module or package 'foo' outside of SciPy and then move it > > into SciPy at a later date. However, every SciPy sub-package that referred > > to foo prior to its inclusion in SciPy now has to be updated from 'import > > foo' to 'import scipy.foo'. These kind of issues make it very painful and > > time consuming to rearrange package structures or move modules and > > sub-packages in and out of the package. > > There are basic python scripts which do this painlessly. If you're really > working on a large project, there's a project architect which normally would > take care of such things, and for whom this should not be a too much of a > problem. > > > > Simplifying this will improves > > package development. > > > > > I'm personnally against anything that enlarges the search path uselessly; > > > > Hopefully I've explained why it is useful for complex packages. > > Python helps in many areas, but expecting it to palliate for the package design > and architecture flaws that inexorably surface anytimes something non-trivial is > developped, might be somehow at the edge. Python has not yet replaced the need > for relevant software architects. > > > > > > because the obvious reason of increased name space collision, increased > > > run-time overhead etc... > > > > I'm missing something here because I don't understand why this increases > > name space collision. If the objection is to the fact that SciPy can have a > > version of Numeric in it that masks a Numeric installed in site-packages, I > > guess I consider this a feature, not a bug. > > Actually, it is normally worse than a bug: it is a source of bug tomorrow in > your application - of all the bugs you'll have when your programmer will be > confusing the two Numeric packages, as well as all the mainteance and > integration problems you'll have down the line -. > > But by then, hopefully for you, you'll be somewhere else... The sad reality of > most projects :(( > > > Afterall, this is already the > > behavior for single level packages, extending it to multi-level packages > > seems natural. If this isn't your objection, please explain. > > > > The current runtime overhead isn't so bad. Prabhu sent me a few numbers on > > the SciPy import (which contains maybe 10-15 nested packages). I attached > > them below -- the overhead is less than 10%. It should be negligible for > > standard modules as only packages are really affected (right Prabhu?). > > And that's how, when you cumulate of the overheads for all new features, you get > potenially +100-200% overhead on the new releases. > Albeit all the efforts of the Python team, Python 2.0 is up to 70% slower than > python 1.5.2; Python 2.1.1 is up to 30% slower than python 2.0, and so on... > So, +10% on only such a minor features is anything but negligible :((( > > FG
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4