Introduction Overview Demo Challenges Conclusion Quickref: a Stress Test for Texinfo Didier Verna EPITA / LRDE didier@lrde.epita.fr lrde/~didier @didierverna didier.verna in/didierverna e TUG 2019 E
Introduction Introduction Quickref: a Stress Test for Texinfo / TUG 2019 – Didier Verna Overview 2/19 Challenges Conclusion Demo ◮ Common Lisp: Social / Community Aspects ◮ The most expressive and extensible language (homoiconicity) ◮ Drawbacks: technical social challenges ◮ Individualism ◮ (Too) Many difgerent solutions for every problem ◮ Quality diffjcult to assert ◮ Many of them ad-hoc or 80% fjnished ◮ Lack of documentation ◮ Consolidation Efgorts ◮ Websites, Resources (guides, tutorials, wikis etc. ) ◮ ASDF, Quicklisp ◮ Introducing Quickref ◮ Global automatic documentation project for Quicklisp libraries ◮ <don> Reference manuals � = user manuals </don>
Introduction Overview Demo Challenges Conclusion Outline System Overview Demonstration Challenges Conclusion & Perspectives Quickref: a Stress Test for Texinfo / TUG 2019 – Didier Verna 3/19
Introduction Overview Demo Challenges Conclusion Outline System Overview Demonstration Challenges Conclusion & Perspectives Quickref: a Stress Test for Texinfo / TUG 2019 – Didier Verna 4/19
Introduction Overview Demo Challenges Conclusion Features Quickref: a Stress Test for Texinfo / TUG 2019 – Didier Verna 5/19 ◮ 2000 or so libraries ◮ Public website: quickref.common-lisp.net ◮ Personal copy: Docker image / Lisp REPL ◮ Private website: local installation only
Introduction Overview Demo Challenges Conclusion Documentation Extraction system components, packages, constants, variables, macros, functions, methods, structures, classes, types, etc. Quickref: a Stress Test for Texinfo / TUG 2019 – Didier Verna 6/19 ◮ Distribution (README fjles etc. ) ◮ ASDF metadata (author, description, repository, etc. ) ◮ Language-level documentation (docstrings) ◮ The rest (software components) ◮ Code Walking (lightweight but very diffjcult) ◮ Introspection (easier but requires loading)
Introduction Overview :license "BSD" :source-control "https://github.com/didierverna/declt" :homepage "http://www.lrde.epita.fr/~didier/software/lisp/" :mailto "didier@didierverna.net" :author "Didier Verna" :description "A reference manual generator for Common Lisp libraries" (asdf:defsystem :net.didierverna.declt ASDF metadata 6/19 Quickref: a Stress Test for Texinfo / TUG 2019 – Didier Verna functions, methods, structures, classes, types, etc. Documentation Extraction system components, packages, constants, variables, macros, Conclusion Challenges Demo ...) ◮ Distribution (README fjles etc. ) ◮ ASDF metadata (author, description, repository, etc. ) ◮ Language-level documentation (docstrings) ◮ The rest (software components) ◮ Code Walking (lightweight but very diffjcult) :long-name "Documentation Extractor from Common Lisp to Texinfo" ◮ Introspection (easier but requires loading)
Introduction system components, packages, constants, variables, macros, BODY should render on *standard-output*." NAME is escaped for Texinfo prior to rendering. "Execute BODY within a @defvr {Constant} NAME environment. (defmacro @defconstant (name &body body) Documentation strings 6/19 Quickref: a Stress Test for Texinfo / TUG 2019 – Didier Verna functions, methods, structures, classes, types, etc. `(@defvr "Constant" ,name ,@body)) Documentation Extraction Overview Conclusion Challenges Demo ◮ Distribution (README fjles etc. ) ◮ ASDF metadata (author, description, repository, etc. ) ◮ Language-level documentation (docstrings) ◮ The rest (software components) ◮ Code Walking (lightweight but very diffjcult) ◮ Introspection (easier but requires loading)
Introduction Declt Quickref: a Stress Test for Texinfo / TUG 2019 – Didier Verna Overview foo.html Makeinfo foo.texi 7/19 foo/ Quicklisp Toolchain Conclusion Challenges Demo ◮ Why Texinfo? ◮ Well suited to technical documentation (reference manual) ◮ Easily converted (PDF, HTML, Info, etc. ) ◮ Built-in index / anchoring / cross-reference facility ◮ Declt: Introspection ◮ Compilation / loading (of dependencies) may be required ◮ Avoid loading 2000 libraries in the same Lisp image! ◮ Run Declt in external processes ◮ Makeinfo: Perl/C script ◮ Ditto ◮ Quickref: Additional glue + loop over all Quicklisp libraries
Introduction Declt Quickref: a Stress Test for Texinfo / TUG 2019 – Didier Verna HTML Files Makeinfo Makeinfo Overview Texinfo Files Makeinfo Declt Declt Library Pool Performance Conclusion Challenges Demo 8/19 ◮ Sequential Processing ◮ Absolute worst-case sequential scenario: 7h ◮ Typical scenario: 2h ◮ Parallel Processing + scheduling algorithm ◮ 4x performance improvement
Introduction Overview Demo Challenges Conclusion Outline System Overview Demonstration Challenges Conclusion & Perspectives Quickref: a Stress Test for Texinfo / TUG 2019 – Didier Verna 9/19
Introduction Overview Demo Challenges Conclusion Demonstration English Conjugation Point demo break demo breaks Quickref: a Stress Test for Texinfo / TUG 2019 – Didier Verna 10/19 � =
Introduction Overview Demo Challenges Conclusion Outline System Overview Demonstration Challenges Conclusion & Perspectives Quickref: a Stress Test for Texinfo / TUG 2019 – Didier Verna 11/19
Introduction A Stress Test for Texinfo Quickref: a Stress Test for Texinfo / TUG 2019 – Didier Verna Overview 12/19 Conclusion Challenges Demo ◮ Scalability ◮ 2000 or so libraries ◮ Dependency management ◮ Foreign dependencies ◮ Library / documentation size ◮ Texinfo Figures ◮ File sizes: 7Ko – 15Mo (x2 HTML) ◮ Lines of code: 364 – 285,020 ◮ Index entries: 14 – 44,500 ◮ Processing time: 0.3s – 1, 38s
Introduction :author "Didier Verna and Antoine Martin" Quickref: a Stress Test for Texinfo / TUG 2019 – Didier Verna public website… Social incentive: people don’t like their work to look bad on my :author ("Didier Verna" "Antoine Martin") :author "D. Verna Antoine \"Joe Cool\" Martin" :author "D. Verna Antoine E Martin" :author "Didier Verna Antoine Martin" :author "Didier Verna, Antoine Martin" :author "<didier@lrde.epita.fr>" Overview :author "didier@lrde.epita.fr" :author "Didier Verna didier@lrde.epita.fr" :author "Didier Verna <didier@lrde.epita.fr>" :author "Didier Verna" Metadata Format Underspecifjcation Conclusion Challenges Demo 13/19
Introduction Jianshi Huang, Quickref: a Stress Test for Texinfo / TUG 2019 – Didier Verna public website… Social incentive: people don’t like their work to look bad on my <maul.mike@gmail.com>" Mike Maul Author Post MSI CLML Contribution: Kuroda Hisao Abe Seika, Fujii Ryo, Abe Yusuke, Overview Tada Masashi, Naganuma Shigeta, Salvi Péter, Original Authors: :author" Metadata Format Underspecifjcation Conclusion Challenges Demo 13/19
Introduction Overview Demo Challenges Conclusion Metadata Format Underspecifjcation :author "(let ((n \"Christoph-Simon Senjak\")) ~ (format nil \"~A <~C~C~C~C~A>\" ~ n (elt n 0) (elt n 10) (elt n 16) ~ #\\@ \"uxul.de\"))" public website… Quickref: a Stress Test for Texinfo / TUG 2019 – Didier Verna 13/19 ◮ Social incentive: people don’t like their work to look bad on my
Introduction (setf context-hyperlinksp) BOOL CONTEXT Quickref: a Stress Test for Texinfo / TUG 2019 – Didier Verna [ doc.lisp ], page 24, (fjle) Source Package [ net.didierverna.declt ], page 29, Access CONTEXT ’s hyperlinksp fmag. [Function] [Function] Overview context-hyperlinksp CONTEXT Example 1: accessors Defjnitions Grouping Conclusion Challenges Demo 14/19
Introduction [ doc.lisp ], page 24, (fjle) Quickref: a Stress Test for Texinfo / TUG 2019 – Didier Verna Source [ asdf.lisp ], page 26, (fjle) Render MODULE ’s documentation in CONTEXT . document ( MODULE module ) CONTEXT Source [ asdf.lisp ], page 26, (fjle) Render SYSTEM ’s documentation in CONTEXT . document ( SYSTEM system ) CONTEXT Methods Source Overview Package [ net.didierverna.declt ], page 29, Render ITEM ’s documentation in CONTEXT . [Generic Function] document ITEM CONTEXT Example 2: generic functions Defjnitions Grouping Conclusion Challenges Demo 14/19
Introduction Overview Quickref: a Stress Test for Texinfo / TUG 2019 – Didier Verna @defvrx {Symbol Macro} foo ... @deffn {Function} foo ... @deffnx {Compiler Macro} ... @deffn {Function} ... Defjnitions Grouping Conclusion Challenges Demo 14/19 ◮ Only use the low level interface: @deffn , @defvr , etc. ◮ Environment nesting → unwanted indentation ◮ Heterogeneous @def... / @def...x prohibited ◮ Heterogeneous categories authorized ◮ Remaining Limitations ◮ Only 9 fjxed canonical categories ◮ Some distinctions arguable ( e.g. typed vs. untyped) ◮ Heterogeneous mixing still prohibited
Recommend
More recommend
