One of the parts of the Adobe PostScript interpreter source that
I've been taking inspiration from is designing the initial state
of the virtual memory (the systemdict holding all the operators
and other initialized data structures) to be cached and loaded
from a file.
In my PostScript clone xpost I really tried to make this feature
work and I never quite managed it. Worse, mine creates multiple
image files every time it runs and just leaves them cluttering up
the working directory. [...]
One of the parts of the Adobe PostScript interpreter source that I've
been taking inspiration from is designing the initial state of the
virtual memory (the systemdict holding all the operators and other initialized data structures) to be cached and loaded from a file.
In my PostScript clone xpost I really tried to make this feature work
and I never quite managed it. Worse, mine creates multiple image
files every time it runs and just leaves them cluttering up the working directory.
The first problem is pointers. The memory image has to use integer
offset and things instead of pointers. Function pointers for operators
have to be mediated through a table that lives outside of the image.
And it's coordinating the operator codes across the memory image
and the compiled code that leads to coupling the two with a version
number or password or something. If you change the ordering of
operators in the source code, the recompiled program will not work
with an old image.
Then, some amount of the top-level interpreter state needs to be
stashed in a known location (Adobe's code uses a struct VMInfo for
this top-level data).
What other considerations are there in designing such a thing?
In xpost there's a tiny issue of pointers to posix objects related
to file globbing. But I don't know how much I really need to worry
about cleaning those out of the image. If you're trying to resume
an image that stopped in the middle of a file globbing loop, then ...
I suppose I should explicitly disavow any guarantees about the
results in the documentation.
One of the parts of the Adobe PostScript interpreter source that I've
been taking inspiration from is designing the initial state of the
virtual memory (the systemdict holding all the operators and other initialized data structures) to be cached and loaded from a file.
In my PostScript clone xpost I really tried to make this feature work
and I never quite managed it. Worse, mine creates multiple image
files every time it runs and just leaves them cluttering up the working directory.
The first problem is pointers. The memory image has to use integer
offset and things instead of pointers. Function pointers for operators
have to be mediated through a table that lives outside of the image.
And it's coordinating the operator codes across the memory image
and the compiled code that leads to coupling the two with a version
number or password or something.
If you change the ordering of
operators in the source code, the recompiled program will not work
with an old image.
Then, some amount of the top-level interpreter state needs to be
stashed in a known location (Adobe's code uses a struct VMInfo for
this top-level data).
What other considerations are there in designing such a thing?
In xpost there's a tiny issue of pointers to posix objects related
to file globbing. But I don't know how much I really need to worry
about cleaning those out of the image. If you're trying to resume
an image that stopped in the middle of a file globbing loop, then ...
I suppose I should explicitly disavow any guarantees about the
results in the documentation.
Disclaimer : I don't know postscript so I'm kind of guessing here. Also
it might have made sense to crosspost this on comp.lang.postscript .
On Fri, 23 Dec 2022 21:16:38 -0800 (PST)
luserdroog <mij...@yahoo.com> wrote:
One of the parts of the Adobe PostScript interpreter source that I'veWhat for ?
been taking inspiration from is designing the initial state of the
virtual memory (the systemdict holding all the operators and other initialized data structures) to be cached and loaded from a file.
In my PostScript clone xpost I really tried to make this feature workImages shouldn't be in the working directory , they should be at a fixed location unless the user has specified otherwise in some way.
and I never quite managed it. Worse, mine creates multiple image
files every time it runs and just leaves them cluttering up the working directory.
The first problem is pointers. The memory image has to use integerDon't the operators have names ? Why can't you use a hash table saying
offset and things instead of pointers. Function pointers for operators have to be mediated through a table that lives outside of the image.
And it's coordinating the operator codes across the memory image
and the compiled code that leads to coupling the two with a version
number or password or something.
that the code corresponding to operator XYZ is at location N ? I don't
see what the ordering of the operators in some source code would have anything to do with it.
If you change the ordering ofSource code of what ?
operators in the source code, the recompiled program will not work
with an old image.
Then, some amount of the top-level interpreter state needs to beWhy does interpreter state need to be stored ?
stashed in a known location (Adobe's code uses a struct VMInfo for
this top-level data).
What other considerations are there in designing such a thing?I don't know but Common Lisp implementations have the facility to save an image including all the new stuff the user has defined while running the image which is the point. If you google for "X save image" where X would be some Common Lisp implementation like SBCL or clisp or ECL , you will get matches. I have no idea how useful this will be to your own goals but it's an
idea.
In xpost there's a tiny issue of pointers to posix objects relatedI do think that a cleaner option would be cleaning all objects whose operation depends on external state (i.e. state outside the control of the interpreter itself) as opposed to relying on user diligence. If you can't remove the (pointers to) objects then surely they can have a flag which says that the object has been invalidated and if the user tries to use an invalidated object , they would get an error message. So when saving an image
to file globbing. But I don't know how much I really need to worry
about cleaning those out of the image. If you're trying to resume
an image that stopped in the middle of a file globbing loop, then ...
I suppose I should explicitly disavow any guarantees about the
results in the documentation.
, you would traverse all objects (which I assume you have to do anyway) and set the "invalidate" flag for all objects dealing with external state.
This doesn't sound any interpreter I've ever used. Usually there is the intepreter itself - a executable file image; the program to be run, and that's it.
Everything needed is created in-memory and stays there; you don't
attempt to write that stuff to a file.
What is the special requirement here? Is there a large program prelude involved for PostScript, which you don't want to have to repeat each
time it runs a script, so you want cache that state?
Does it need to be able to recover from an aborted script?
Or is there
other state that needs to be persistent across activations? In short,
what is the problem to be solved that might need saving a memory image
to disk and reloading later?
On Saturday, December 24, 2022 at 4:34:00 AM UTC-6, Bart wrote:
Does it need to be able to recover from an aborted script?
That's something I want my clone to do. Or at least get some debugging
info out of it. I'd love to be able to get bug reports with the exact state of the interpreter at the point of the problem and have the tools to diagnose
and fix it. That's the pie-in-the-sky dream anyway.
On Saturday, December 24, 2022 at 7:25:30 AM UTC-6, Spiros Bousbouras wrote:
Disclaimer : I don't know postscript so I'm kind of guessing here. Also
it might have made sense to crosspost this on comp.lang.postscript .
On Fri, 23 Dec 2022 21:16:38 -0800 (PST)
luserdroog <mij...@yahoo.com> wrote:
The first problem is pointers. The memory image has to use integer offset and things instead of pointers. Function pointers for operators have to be mediated through a table that lives outside of the image.Don't the operators have names ? Why can't you use a hash table saying that the code corresponding to operator XYZ is at location N ? I don't
And it's coordinating the operator codes across the memory image
and the compiled code that leads to coupling the two with a version number or password or something.
see what the ordering of the operators in some source code would have anything to do with it.
The operator objects are created dynamically and allocated sequential
codes at "ram initialization time". Under modern operating systems you
can't reliably store function pointers across separate invocations of the (interpreter) program because of ASLR. cf.
https://stackoverflow.com/questions/18040735/will-statically-linked-functions-have-the-same-pointer-values-across-runs-on-the
If you change the ordering ofSource code of what ?
operators in the source code, the recompiled program will not work
with an old image.
The C code that implements the interpreter.
Then, some amount of the top-level interpreter state needs to beWhy does interpreter state need to be stored ?
stashed in a known location (Adobe's code uses a struct VMInfo for
this top-level data).
I'm not sure what I've failed to describe. The goal is for the entire state of execution to be suspended to disk and resumed from the same point
after power cycling. Or, for a hosted interpreter that runs on an OS, you'd resume by running the interpreter program again with an option to resume
the suspended session.
What other considerations are there in designing such a thing?I don't know but Common Lisp implementations have the facility to save an image including all the new stuff the user has defined while running the image which is the point. If you google for "X save image" where X would be
some Common Lisp implementation like SBCL or clisp or ECL , you will get matches. I have no idea how useful this will be to your own goals but it's an
idea.
Yes, I think this is probably the closest examples I'm likely to find. I think
emacs does something similar.
On Saturday, December 24, 2022 at 4:34:00 AM UTC-6, Bart wrote:
Or is there
other state that needs to be persistent across activations? In short,
what is the problem to be solved that might need saving a memory image
to disk and reloading later?
I suppose there are just 2 or 3 use cases I can think of:
1. quick loading of the interpreter without having to run through all of it's
dynamic initialization of all it's many various data structures.
2. storing fonts and stuff into this image.
3. debugging an image after a crash.
On Sat, 24 Dec 2022 11:38:22 -0800 (PST)
luserdroog <mij...@yahoo.com> wrote:
On Saturday, December 24, 2022 at 4:34:00 AM UTC-6, Bart wrote:
Or is there
other state that needs to be persistent across activations? In short, what is the problem to be solved that might need saving a memory image to disk and reloading later?
I suppose there are just 2 or 3 use cases I can think of:
1. quick loading of the interpreter without having to run through all of it'sOf course for 3 to be possible , the interpreter would have to regularly save
dynamic initialization of all it's many various data structures.
2. storing fonts and stuff into this image.
3. debugging an image after a crash.
state as opposed to waiting for a user command to do so.
luserdroog <mij...@yahoo.com> writes:
One of the parts of the Adobe PostScript interpreter source that
I've been taking inspiration from is designing the initial state
of the virtual memory (the systemdict holding all the operators
and other initialized data structures) to be cached and loaded
from a file.
In my PostScript clone xpost I really tried to make this feature
work and I never quite managed it. Worse, mine creates multiple
image files every time it runs and just leaves them cluttering up
the working directory. [...]
If you want help, define the problem completely and clearly enough
so people who don't know anything about PostScript can understand
what it is you want to accomplish.
On Saturday, December 24, 2022 at 3:05:54 AM UTC-6, Tim Rentsch wrote:
luserdroog <mij...@yahoo.com> writes:
One of the parts of the Adobe PostScript interpreter source that
I've been taking inspiration from is designing the initial state
of the virtual memory (the systemdict holding all the operators
and other initialized data structures) to be cached and loaded
from a file.
In my PostScript clone xpost I really tried to make this feature
work and I never quite managed it. Worse, mine creates multiple
image files every time it runs and just leaves them cluttering up
the working directory. [...]
If you want help, define the problem completely and clearly enough
so people who don't know anything about PostScript can understand
what it is you want to accomplish.
You right. I didn't realize how scatterbrained and incomplete my
post was. From the start, I think I need to separate the new
features from the bug fix. And prioritize the latter.
The bug.
When xpost begins executing it creates two mmap'ed files named
gmemXXXXXX and lmemXXXXXX (and if logging is enabled there's also
a third xdumpXXXXXX file created). Each file has its own unique
XXXXXX string. So when there's more than one set of these files
in the directory it'd be very difficult to know which ones are
related.
So what I need is a function like tmpfile() that takes a set of
filename templates and fills in each with the same unique XXXXXXs
so that none of them conflict with existing files. [...]
The language design issue I'm really fuzzy on is what options to
have for the interpreter so it can clean up after itself -- or
preserve the evidence -- as needed. As a first draft, does the
following seem reasonable?
xpost --clean-up % the default
xpost --preserve-temps % or --keep-memory-files ?
For the --clean-up option, there's a programming question of
whether to remove() the files at exit or just use an anonymous
mmap() to begin with.
luserdroog <mij...@yahoo.com> writes:[snip]
On Saturday, December 24, 2022 at 3:05:54 AM UTC-6, Tim Rentsch wrote:
luserdroog <mij...@yahoo.com> writes:
One of the parts of the Adobe PostScript interpreter source that
I've been taking inspiration from is designing the initial state
of the virtual memory (the systemdict holding all the operators
and other initialized data structures) to be cached and loaded
from a file.
In my PostScript clone xpost I really tried to make this feature
work and I never quite managed it. Worse, mine creates multiple
image files every time it runs and just leaves them cluttering up
the working directory. [...]
If you want help, define the problem completely and clearly enough
so people who don't know anything about PostScript can understand
what it is you want to accomplish.
You right. I didn't realize how scatterbrained and incomplete my
post was. From the start, I think I need to separate the new
features from the bug fix. And prioritize the latter.
The bug.
I'm still hoping to see a summary description (concise but complete)
of all the different parts of the environment, what operations need
to be performed on the different kinds of "snapshots", and which
parts of the environment you think need to go in each kind of
snapshot. I have done some work in and on PostScript in the past,
but that was some years ago, so the safest assumption is that I
don't remember anything, and also may not have understood what all
the different parts are even when I was fully immersed in PostScript
work.
what operations need
to be performed on the different kinds of "snapshots", and which
parts of the environment you think need to go in each kind of
snapshot.
One of the parts of the Adobe PostScript interpreter source that I've
been taking inspiration from is designing the initial state of the
virtual memory (the systemdict holding all the operators and other initialized data structures) to be cached and loaded from a file.
In my PostScript clone xpost I really tried to make this feature work
and I never quite managed it. Worse, mine creates multiple image
files every time it runs and just leaves them cluttering up the working directory.
On Sunday, December 25, 2022 at 8:58:33 PM UTC-6, Tim Rentsch wrote:[...]
I'm still hoping to see a summary description (concise but complete)
of all the different parts of the environment, [...]
[.. long and detailed description ..]
luserdroog <mij...@yahoo.com> writes:
On Sunday, December 25, 2022 at 8:58:33 PM UTC-6, Tim Rentsch wrote:[...]
I'm still hoping to see a summary description (concise but complete)
of all the different parts of the environment, [...]
[.. long and detailed description ..]
A blizzard of information... more than is needed in some areas,
and (I suspect) less than is needed in others.
Part of the reason for asking for a _concise_ but complete
summary description is to get you to organize the information so
it can be so presented. Going through the effort of organizing
the information in this way should go a long way towards helping
you implement the state-saving functionality.
Do these comments help you see what I'm getting at?
luserdroog <mij...@yahoo.com> writes:
On Sunday, December 25, 2022 at 8:58:33 PM UTC-6, Tim Rentsch wrote:[...]
I'm still hoping to see a summary description (concise but complete)
of all the different parts of the environment, [...]
[.. long and detailed description ..]
A blizzard of information... more than is needed in some areas,
and (I suspect) less than is needed in others.
Part of the reason for asking for a _concise_ but complete
summary description is to get you to organize the information so
it can be so presented. Going through the effort of organizing
the information in this way should go a long way towards helping
you implement the state-saving functionality.
For example, any state that is held in integer data types can all
be lumped together, because saving integers is well-understood
and pretty easy.
Conversely, function pointers need special care, because they
cannot just be stored directly.
The long description given mentions "objects" but as far as I can
tell what an "object" (Xpost_Object?) is is never defined.
Are references to objects done with pointers or by means of an
object table? If there were an object table that would greatly
simplify (probably) the state-saving operations.
You mention "operations" but don't say what an operation is.
Also, there is some amount of state for the garbage collector.
Probably that state does not need to be (directly) saved for
a state-saving operation. What information is important to save,
and what information is incidental and can be ignored?
Do these comments help you see what I'm getting at?
On Thursday, December 29, 2022 at 8:06:37 PM UTC-6, Tim Rentsch wrote:
luserdroog <mij...@yahoo.com> writes:
On Sunday, December 25, 2022 at 8:58:33 PM UTC-6, Tim Rentsch wrote:
[...]
I'm still hoping to see a summary description (concise but complete)
of all the different parts of the environment, [...]
[.. long and detailed description ..]
A blizzard of information... more than is needed in some areas,
and (I suspect) less than is needed in others.
Part of the reason for asking for a _concise_ but complete
summary description is to get you to organize the information so
it can be so presented. Going through the effort of organizing
the information in this way should go a long way towards helping
you implement the state-saving functionality.
For example, any state that is held in integer data types can all
be lumped together, because saving integers is well-understood
and pretty easy.
Conversely, function pointers need special care, because they
cannot just be stored directly.
The long description given mentions "objects" but as far as I can
tell what an "object" (Xpost_Object?) is is never defined.
Are references to objects done with pointers or by means of an
object table? If there were an object table that would greatly
simplify (probably) the state-saving operations.
You mention "operations" but don't say what an operation is.
Also, there is some amount of state for the garbage collector.
Probably that state does not need to be (directly) saved for
a state-saving operation. What information is important to save,
and what information is incidental and can be ignored?
Do these comments help you see what I'm getting at?
Yes. Let me try again having cleaned and straightened up my bifocals.
From the top level, and abstracting away all the fiddlybits, the
whole interpreter is just a collection of execution contexts.
Since it's just a simple round robin scheduling algorithm, it
doesn't even really need to remember the current context to resume
execution.
Interpreter
collection of Contexts
Next, an execution context has a global memory and a local memory,
where there is a rule that global memory ought not to contain any
references to things in local memory. So, the global memory can be considered self-contained with the local memory forming a shell
around it.
Context
Global Memory
Local Memory
a collection of integers (flags, offsets into local memory)
a collection of Objects (current object, window device, window
device event handler)
[another 150 lines]
Sysop: | DaiTengu |
---|---|
Location: | Appleton, WI |
Users: | 906 |
Nodes: | 10 (0 / 10) |
Uptime: | 239:25:37 |
Calls: | 12,039 |
Calls today: | 2 |
Files: | 186,478 |
Messages: | 2,215,266 |