

This directory contains a new emacs-lisp byte-compiler.
This file describes the compiler and its installation procedure.

This compiler is included with Lucid GNU Emacs, a divergent fork of Emacs 19,
and will probably be included in FSF's Emacs 19 release, someday.  For more
information about Lucid GNU Emacs, see the README on labrea.stanford.edu in
the directory /pub/gnu/lucid/.

The new compiler has the following improvements:

 + optimization of compiled code:
   - removal of unreachable code;
   - removal of calls to side-effectless functions whose return-value
     is unused;
   - compile-time evaluation of safe constant forms, such as (consp nil)
     and (ash 1 6);
   - open-coding of literal lambdas;
   - peephole optimization of emitted code;
   - trivial functions are left uncompiled for speed.
 + support for inline functions;
 + compile-time evaluation of arbitrary expressions;
 + compile-time warning messages for:
   - functions being redefined with incompatible arglists;
   - functions being redefined as macros, or vice-versa;
   - functions or macros defined multiple times in the same file;
   - functions being called with the incorrect number of arguments;
   - functions being called which are not defined globally, in the 
     file, or as autoloads;
   - assignment and reference of undeclared free variables;
   - various syntax errors;
 + correct compilation of nested defuns, defmacros, defvars and defsubsts;
 + correct compilation of top-level uses of macros;
 + the ability to generate a histogram of functions called.

It consists of the following files:

 bytecomp.el		The guts of the compiler.  This may be autoloaded.

 byte-optimize.el	The optimization aspects of the compiler.  This is
			loaded as needed.

 bytecomp-runtime.el	New functions and macros that should be around even
			if the byte-compiler proper is not loaded.  This file
			contains the definitions of several macros which are
			advertised interfaces, and should be available to
			interpreted code and compiled code whether the 
			compiler itself is loaded or not.  This  file should
			be loaded by your .emacs file, or (preferably) by
			loadup.el or startup.el.

 disass.el		A disassembler for the new compiler.  The old 
			disassembler no longer works.

 bytecode.c		For Emacs19 only.  This is a replacement byte-code
			interpreter that defines the new byte-codes.
			You *do not need this* to use the new compiler.
			This file does not work in Emacs version 18.

 data.c.patch		For Emacs19 only.  This patch adds a new subr,
			compiled-function-p, which identifies Lisp_Compiled
			objects.

 eval.c.patch		This contains a better version of the macroexpand
			function.  The patch is for v19, but you can easily
			extract the function if you're so moved.

 lread.c.patch		For Emacs19 only.  This patch adds a new read-syntax,
			#[...], for reading Lisp_Compiled objects directly.
			This patch also removes the change to v19 that causes
			numbers beginning with 0 to be read in octal, as that
			causes a huge amount of existing code to break.

 print.c.patch		For Emacs19 only.  This patch adds a new variable,
			print-readably, which, when bound to t, makes lists
			of the form (quote <thing>) to print in the more
			more concise (and faster to read) format of '<thing>.
			Also, when print-readably is true, an error will be
			signalled if there is an attempt to print out an
			unreadable object (the #<...> objects).  This
			is used by the compiler as a way of detecting some
			load-time errors at compile-time.

 make-docfile.c		A version of etc/make-docfile which understands the new
			format used in .elc files.  This will work with V18,
			but is only necessary if you are using V19, and are
			building an emacs which preloads things compiled with
			the new compiler.

To install the new compiler, you may either place the above .el files in the
standard emacs lisp directory, replacing the existing files called bytecomp.el
and disass.el, or place the above .el files in their own directory, and add
that directory to the front of your load-path.

It is important that you not mix up the old and new compilers; the new
disass.el won't work with the old compiler, and the old disass.el will
screw up the new compiler.

It is not necessary to install any of the patch files into your emacs; in 
fact, it would be a lot of work to do so unless you are running a prerelease
of version 19.

The new compiler can be compiled with the old compiler, but that's probably
not the best idea.  To compile the new compiler with itself, simply do this:

 M-x load-file bytecomp-runtime.el
 M-x load-file bytecomp.el
 M-x load-file byte-optimize.el
 M-x byte-compile-and-load-file bytecomp-runtime.el
 M-x byte-compile-and-load-file bytecomp.el
 M-x byte-compile-and-load-file byte-optimize.el
 M-x byte-compile-and-load-file disass.el

That is, first load all of the compiler interpreted, and then compile it with
itself.

The byte-compiler can be configured to produce files which depend on the 
existence of the new byte-codes and read-syntax, or not.  The defaults should
be correct in whichever version of emacs you happen to be running.

If you're not running Epoch, you will get some warnings of the form

  While compiling toplevel forms in file bytecomp.el:
    ** reference to free variable epoch::version

Ignore them.  Since Epoch doesn't follow the normal Emacs conventions for what
the value of emacs-version looks like, we have to look at epoch::version as
well to decide whether we're running under a v18-vintage emacs, or a v19 one.

Questions, comments, and suggestions are more than welcome!

(But please do not flood me with requests for a prerelease version of emacs19.
Complaints about the filenames being longer than 14 characters will be 
gleefully ignored.)

Enjoy,
	  -- Jamie Zawinski <jwz@lucid.com>

------------------------------------------------------------------------------
The following variables may be used to customize the compiler's behavior:

byte-compile-verbose	Whether to report the function currently being 
			compiled in the minibuffer (default t).

byte-optimize		Whether to do optimizations; this may be t, nil,
			'source, or 'byte (default t).

byte-optimize-log	Whether to report (in excruciating detail) exactly
			which optimizations have been made.  This may be t,
			nil, 'source, or 'byte (default nil).

byte-compile-error-on-warn	Whether to stop compilation when a warning is
				produced (default nil).

byte-compile-delete-errors	Whether the optimizer may delete calls or 
				variable references that are side-effect-free
				except that they may signal an error (default
				t).  Set this to nil if you use the abysmal
				style of doing type-assertions this way.

byte-compile-generate-call-tree	Whether to generate a histogram of function
				calls.  This can be useful for finding unused
				functions, as well as simple performance
				metering.

byte-compile-warnings	List of warnings to issue, or t.  May contain
			free-vars   references to variables not in the current
			            lexical scope
			unresolved  calls to unknown functions
			callargs    function calls with args that don't match
				    the function's definition
			redefine    function cell redefined from a macro to a
				    function or vice versa, or redefined to 
				    take incompatible args
			This defaults to nil in -batch mode, which is slightly
			faster, and t otherwise.

byte-compile-emacs18-compatibility	Whether the compiler should generate
			.elc files which can be loaded into generic emacs 18's
			which don't have the file bytecomp-runtime.el loaded.

byte-compile-generate-emacs19-bytecodes	Whether to generate bytecodes which
			exist only in emacs19.  This is a more extreme step
			than setting emacs18-compatibility to nil, because
			there is no elisp you can load into an emacs18 to make
			files compiled this way work.

byte-compile-single-version	Normally the byte-compiler will consult the
			above two variables at runtime, but if this variable 
			is true before the compiler itself is loaded/compiled,
			then the runtime checks will not be made, and
			compilation will be slightly faster.  To use this, 
			start up a fresh emacs, set this to t, and repeat the
			compiler-compilation steps above.  Make sure you set
			this before loading the .el files, or you will lose.

elisp-source-extension-re	Regexp for the extension of elisp source-files;
				see also the function byte-compile-dest-file.

byte-compile-overwrite-file	If nil, old .elc files are deleted before the
			new one is saved, and .elc files will have the same 
			modes as the corresponding .el file.  Otherwise, 
			existing .elc files will simply be overwritten, and 
			the existing modes will not be changed.  If this
			variable is nil, then an .elc file which is a
			symbolic link will be turned into a normal file, 
			instead of the file which the link points to being
			overwritten.

Most of the above parameters can also be set on a file-by-file basis; see
the documentation of the `byte-compiler-options' macro.

------------------------------------------------------------------------------
Miscellaneous new features:

  o  The form `defsubst' is just like `defun', except that the function
     generated will be open-coded in compiled code which uses it.  This
     means that no function call will be generated, it will simply be
     spliced in.  Elisp functions calls are very slow, so this can be a
     big win.

     You can generally accomplish the same thing with `defmacro', but in
     that case, the defined procedure can't be used as an argument to
     mapcar, etc.

     Remember that excessive inlining means your .elc file size and memory
     consumption can go way up.

  o  You can make a given function be inline even if it has already been
     defined with `defun' by using the `proclaim-inline' form like so:
	(proclaim-inline my-function)
     This is, in fact, exactly what `defsubst' does.  To make a function no
     longer be inline, you must use `proclaim-notinline'.  Beware that if
     you define a function with `defsubst' and later redefine it with 
     `defun', it will still be open-coded until you use proclaim-notinline.

  o  You can also open-code one particular call to a function without
     open-coding all calls.  Use the `inline' form to do this, like so:

		(inline (foo 1 2 3))	;; `foo' will be open-coded
	or...
		(inline			;;  `foo' and `baz' will be 
		 (foo 1 2 3 (bar 5))	;; open-coded, but `bar' will not.
		 (baz 0))

  o  It is possible to open-code a function in the same file it is defined
     in without having to load that file before compiling it.  the
     byte-compiler has been modified to remember function definitions in
     the compilation environment in the same way that it remembers macro
     definitions.

  o  Forms like ((lambda ...) ...) are open-coded.

  o  The form `eval-when-compile' is like progn, except that the body
     is evaluated at compile-time.  When it appears at top-level, this
     is analogous to the Common Lisp idiom (eval-when (compile) ...).
     When it does not appear at top-level, it is similar to the
     Common Lisp #. reader macro (but not in interpreted code.)
     If you're thinking of using this, consider whether `provide' and
     `require' might do the job as well.

  o  The form `eval-and-compile' is similar to eval-when-compile, but
     the whole form is evalled both at compile-time and at run-time.
     If you're thinking of using this, consider whether `provide' and
     `require' might do the job as well.

  o  The command `byte-compile-and-load-file' does what you'd think.

  o  The command `elisp-compile-defun' is analogous to `eval-defun'.
     I bind this to ^C^C and bind `eval-defun' to ^C^E.

  o  If you run `byte-compile-file' on a filename which is visited in a 
     buffer, and that buffer is modified, you are asked whether you want
     to save the buffer before compiling.

------------------------------------------------------------------------------
The following problems will be seen when compiling the Emacs 18.57 source tree
with the new compiler.  (I mention them here only to point out that they are
not bugs in the compiler itself.)

The most obvious problem is that if byte-compile-warnings is t, or a list
containing the symbol 'free-vars, roughly a billion warnings will be produced,
since much of the code in the default emacs library takes advantage of the
fact that emacs-lisp variables have dynamic scope.  I consider it
exceptionally poor style to use dynamic scoping unless absolutely necessary,
and if you agree, then the free-vars warning can be invaluable in tracking
down unintentional variable references and misspellings.

The free-variable warning does not mean the compiler has miscompiled the code;
it is merely a sanity-check.  See the comments at the top of byte-optimize.el 
for some meandering thoughts about lexical scope.

bibtex.el loses because the Sunwindows macro `defmenu' is undefined, and
without knowing that it's a macro, this file looks like it's invoking
strings as functions.  Perhaps this wouldn't lose if it were compiled under
Sunwindows.

While compiling corelate-sequences in file buffer-merge.el:
  ** ifind-longest-subsequence called with 3 arguments, but accepts only 2

While compiling the end of the data in file c++-mode.el:
  ** the function strings-to-char is not known to be defined.
(used in c++-uncomment-region, should be string-to-char.)

While compiling generate-month in file cal.el:
  ** malformed let binding: (first-day-of-month (day-of-week month 1 year) 7)

While compiling doctor-svo in file doctor.el:
  ** malformed let binding: (foo (doctor-subjsearch sent key type) sent)

While compiling rmail-widen-to-current-msgbeg in file rmail.el:
  ** malformed let binding: (unwind-protect (progn (narrow-to-region (rmail-msg
beg rmail-current-message) (point-max)) (goto-char (point-min)) (funcall functi
on)) (narrow-to-region (- (point-max) obeg) (point-max)))

This is bad news: it means that a form which was intended to be inside of
an unwind-protect is really the value of a variable named unwind-protect.
And the protected clause is being ignored.

The file sun-cursors.el cannot be compiled because it exhibits a fundamental
misunderstanding of how macros are supposed to work.  It it written with the
assumption that macroexpand-time and load-time are one and the same, which 
was a bug in the old compiler, which is fixed in this new one.

The completer.el package from Thinking Machines (version 10.3 and older)
exhibits the same fatal assumption.

The file cl.el must be loaded before it can be compiled, because it is a
very circular, incestuous piece of code.  It makes use of the macro `do' 
before defining it; and even if the `do' macro were earlier in the file,
it's expander makes use of functions defined in the same file, which 
are not defined in the compilation environment.

Another thing to realize is that the `gensym' function from cl.el doesn't
really work: when you write an uninterned symbol out to an .elc file, it
looks like an interned symbol, and becomes one when that file is loaded.
So if you've got a function that uses a gensym as a variable name (as any
function which uses the `pop' macro would, for example) that function will
be binding an interned symbol named something like `G_$$5', instead of an
uninterned one.  Gensyms don't exist, and can't until/unless there is some
equivalent of Common Lisp's #: reader macro.
