[lfs-dev] How to add/How i added/ BLFS XML source content into an LFS Book's source tree

[Kevin Buckley via lfs-dev]

Not sure if this was the best list for this posting but the people
on here will know best. It's quite possibly a "Hint", although
as it does touch on the behind the scenes rendering of the
books themslves, there may be interested parties across a
number of the lists, hecne falling back to lfs-dev.

Anyroad, here goes:

As I have just gone through this process, I thought I'd
write it up in case it is of use to anyone else who wants
to have their own copy of a LFS+BLFS book that contains
the packages that their system ends up with.

I've also found that creating my own book from the SVN
sources allows me to extract a full set of commands, into
a text file, that I often end up cutting-and-pasting from,
although mine are not an attempt to automate a build.



From a directory, mine is called SVN, containing the
checked out SVN sources for both books, so

./SVN/BLFS/8.3
./SVN/LFS/8.3


Make a copy of the original LFS sources

cp -a LFS/8.3 LFS/8.3-kmb

cd LFS/8.3-kmb


Alter a couple of settings in the Makefile so as
to keep everything within the existing directory
hierachy, ie, below my top-level SVN


vim Makefile

BASEDIR = ../../8.3-kmb-rendered
DUMPDIR = ../../8.3-kmb-commands


Prepare the new source tree for placing the BLS XML sources into
it in such a way as to allow then to find the files they expect
to at the relative locations they expect to.


mkdir -p chapter10/blfs-packages/blfs/packages/

mkdir -p chapter10/blfs-packages/xincludes

cp -p ../../BLFS/8.3/introduction/important/bootscripts.xml \
chapter10/blfs-packages/blfs/packages/

cp -p ../../BLFS/8.3/general/sysutils/gpm.xml \
chapter10/blfs-packages/blfs/packages/

cp -p ../../BLFS/8.3/networking/textweb/links.xml \
chapter10/blfs-packages/blfs/packages/

cp -p ../../BLFS/8.3/general/genlib/popt.xml \
chapter10/blfs-packages/blfs/packages/

cp -p ../../BLFS/8.3/general/sysutils/pciutils.xml \
chapter10/blfs-packages/blfs/packages/

cp -p ../../BLFS/8.3/general/graphlib/libpng.xml \
chapter10/blfs-packages/blfs/packages/

cp -p ../../BLFS/8.3/postlfs/filesystems/dosfstools.xml \
chapter10/blfs-packages/blfs/packages/

cp -p ../../BLFS/8.3/general/graphlib/freetype2.xml \
chapter10/blfs-packages/blfs/packages/


Even though a UEFI-aware Grub is built with a different
configuration to the LFS Book's version, it make a lot of
sense to use the XML source from Chapter 6 as the basis
for documenting the rebuild, so


cp -p ../../LFS/8.3/chapter06/grub.xml \
chapter10/

vim chapter10/grub.xml


The next three are packages required for a UEFI-aware Grub
2.02 but which are not in BLFS. I based my XML sources on a
"template" created by copying a BLFS XML source for a package
that had very little "build" required. (can't recall which one now)


vim chapter10/efivar.xml

vim chapter10/efibootmgr.xml

vim chapter10/unifont.xml


These next files are general sources that many, if not all
of the BLFS package source files excpect to see, and so
about which the rendering process will compalin if not there.


cp -p ../../BLFS/8.3/general.ent \
../../BLFS/8.3/gnome.ent \
../../BLFS/8.3/packages.ent \
chapter10/blfs-packages/

cp -p ../../BLFS/8.3/xincludes/gtk-doc-rebuild.xml \
../../BLFS/8.3/xincludes/static-libraries.xml \
chapter10/blfs-packages/xincludes


Finally, create a "chapter source" that includes all of
the "from-BLFS" or new packages sources within Chapter 10.

cp -p ../../LFS/8.3/chapter06/chapter06.xml \
chapter10/chapter10.xml

vim chapter10/chapter10.xml


Edit the main Index files so include our Chapter 10


vim index.html


Needed to edit the main Makefile again, as I tried using the "all"
target, so as to get the "command" content extracted, but found
that I didn't have all the utilities required to create a PDF
version.


vim Makefile


Make the target that creates everything we need


make all-but-pdf


Now go and concatenate the individually extracted command files.


pushd ../../8.3-kmb-commands

$ ls chapter10
160-extra-intro 164-popt 167-freetype2 170-efibootmgr
162-gpm 165-pciutils 168-dosfstools 171-unifont
163-links 166-libpng 169-efivar 172-grub


for c in 2 3 4 5 6 7 8 ; do cd chapter0$c ; for s in * ; do
cat $s >> ../ALL-commands.txt ; echo "" >> ../ALL-commands.txt ;
done ; cd .. ; done

for c in 10 ; do cd chapter$c ; for s in * ; do
cat $s >> ../ALL-commands.txt ; echo "" >> ../ALL-commands.txt ;
done ; cd .. ; done


I then copy ALL-commands.txt into my $LFS/sources area
so that I have a copy of it when on the LFS system, prior
to adding in some "beyond LFS" stuff.

Obviously up until the point that you have a mouse to
cut-and-paste with, the benefits of such a file are
small, however, you do always have the abiity to extract
parts of any of it using just the shell utilities (exercise left
for the reader!).


Notes:

Pulling in XML sources from BLFS can sometimes see missing
references, to link targets in BLFS XML sources that haven't
been pulled acroos into the new book's source tree, prevent
the modified book from being rendered.

Fixing up these references can be done in (at least) one of
these two ways.

1) Create a "chapter10/notinstalled.xml" file into which the
various target id-s can be added into an element, which,
for example, could be just a paragraph, so

<sect1 id="ch-not-installed">
<?dbhtml filename="notinstalled.html"?>

<title>Not Installed</title>

<para>Because I have tried to use the BLFS package sections
to initially populate this record of what I built, there are
some link references that need targets:</para>
<para id="curl">There was a reference to the curl package</para>
<para id="lynx">There was a reference to the lynx package</para>
...

One drawback to this approach is that in the rendered HTML text,
you lose the name of the target, for example

Required

libffi-3.2.1 and Python-2.7.13 or Section 10.19, “Not Installed”

although it does allow the new book to be rendered.


2) Edit the BLFS XML source so that the <xref> tags get commented
out and replaced with just the text of their linkend attribute.

Here's an example from the BLFS Freetype2 XML source, where,
because my particular build didn't need the "Recommended"
package "harfbuzz", the latter's source wasn't pulled across
and, so as to avoid complaints about missing links, the original
line

<xref linkend="harfbuzz"/>

simply gets replaced by

<!--xref linkend="harfbuzz"/--> harfbuzz

One benefit of this approach is that the original XML is
maintained within the modified source so, if, in the case
above, the Harfbuzz package was being built later on and
its source XML had been pulled across into the modified
book, then it is an easy matter to restore the link.


There'll be other ways to do this, of course, but this is just
one that has worked for me.


In theory, it might be possible to do an SVN diff so as to
generate a new set of "starting sources" for subsequent LFS
Book releases, but whilst I have done such a diff, so as to
try and identify and verify the changes I've made, I haven't
yet tried using it as a staring off point.


Hoping that someone finds the above useful.
Kevin

PS

It has occured to me whilst writing up what i did, based on
what is effectively my "build a UEFI-aware Grub" chapter
that there might be some milage in producing some kind of
repository of "BLFS chapters" that pull together the various
packages required to install "Package X", along the line of
Bruce's "Basic BLFS" book but a lot smaller and potenially
more easily melded into a single "modified book". I, for example,
have what is effectivley a "Xen chapter" from around LFS 8.0,
lying around, but it is somethingI put together before knowing
what I do now about pulling in the various BLFS sources in a
way that reduces the amount fo work required to get to rendered
(HTML) book.

Just (another) thought though, really.