Merge branch 'dev' into 'master'

2 more papers by Çagdas Bozman

See merge request OCamlPro/www!36

src/content/blog/2012_04_22_april_monthly_report.md

@@ -2,7 +2,7 @@ title=April Monthly Report
 authors=Çagdas Bozman
 date=2013-04-22
 category=OCamlPro
-tags=OCamlPro,OCaml 
+tags=Memory,OCaml,OCamlPro,ocp-indent,Profiling,Tools
 
 This post aims at summarizing the activities of OCamlPro for the past month. As usual, we worked in three main areas: the OCaml toolchain, development tools for OCaml and R&D projects.
 

src/content/blog/2013_04_02_wxocaml_camlidl_and_class_modules.md

@@ -2,7 +2,7 @@ title=wxOCaml, camlidl and Class Modules
 authors=Çagdas Bozman
 date=2013-04-02
 category=Tooling
-tags=OCaml, Opam, Tooling
+tags=OCamlPro,wxocaml,OCaml
 
 Last week, I was bored doing some paperwork, so I decided to hack a little to relieve my mind...
 

src/content/blog/news-from-july.md

+title=News from July 
+authors=Çagdas Bozman
+date=2013-08-5
+category=OCamlPro
+tags=editor,OCaml,OCamlPro,OCamlTop
+
+Once again, here is the summary of our activities for last month. The highlight this month is the release of [ocaml-top](http://www.typerex.org/ocaml-top.html), an interactive editor for education which works well under Windows and that we hope professors all around the world will use to teach OCaml to their students. We are also continuying our work on the improvement of the performance of OCaml, with new inlining heuristics in the compiler and adding multicore support to the runtime.
+
+## Compiler updates
+
+Last month, we started to get very nice results with our compiler performance improvements. First, Pierre Chambart polished the prototype implementation of his new `flamba` intermediate language and he started to get impressive [micro-benchmarks results](http://ocamlpro.com/2013/07/11/better-inlining-progress-report/), with around 20% – 30% improvements on code using exceptions or functors. Following a discussion with our industrial users, he is currently focusing on improving the compilation of local recursive functions such as the very typical:
+
+```ocaml
+let f x y =  
+let rec loop v =  
+… x …  
+loop z  
+in  
+loop x  
+```
+
+A simple and reasonably efficient solutions is to eta-expand the auxiliary function, i.e. add an intermediate function calling the loop with all closure parameters passed as variables. The hard part is to then to add the right arguments to all the call sites: luckily enough the new inlining engine already does that kind of analysis so it can be re-used here. This means that these constructs will be compiled efficiently by the new inlining heuristics.
+
+Second, Luca Saiu has finished debugging the native thread support on top of his [multi-runtime variant of OCaml](https://github.com/lucasaiu/ocaml), which has become quite usable and is pretty robust now. He has tentatively started adding support for `vmthreads` as well, concurrently cleaning up context finalization and solving other minor issues, such as configuration scripts for architectures that do not support the multi-runtime features yet. Then, after writing documentation and running a full pass over the sources to remove debugging stubs and prints which pollute the code after months of low-level experimentation, he is going to prepare patches for discussion and submission to the main OCaml compiler.
+
+Çagdas Bozman continued to improve the implementation of his [profiling tools](https://github.com/cago/ocaml) for both native and byte-code programs. A great output of his recent work is that the location information is much more precise: with very different techniques for native and byte code, the program locations are now uniquely identified. The usability was improved as well, as the profiling location tables are now embedded directly into the programs. He also improved the post-mortem profiling tools to re-type dumped heaps, which also leads to much more accurate profiling information. Çagdas is now actively using these tools on [why3](http://why3.lri.fr/) and he expects to get feedback results very soon to continue to improve his tools.
+
+Finally, Thomas Blanc is still working on whole program analysis for his internship, in order to find possibly uncaught exceptions. The project is moving quite well and the month was spent on analyzing the lambda intermediate representation of the compilation step. With the help of Pierre Chambart, he is working on a [0-CFA library](https://github.com/thomasblanc/ocaml-data-analysis) that should allow to compute the “possible” values for each variable at each point of the program. The idea is to make a directed hypergraph with each hyperedge representing an instruction and each vertex being a state of the program. Then search a fixpoint in the possible values propagated through the graph. This allows the compiler to know everywhere in the program what possible values may be returned or what possible exceptions may be raised. In order to create a well-designed graph, it is needed to create a new intermediate representation that looks like Lambda except (mainly) that every expression gets an identifier. The next step is to specify a hypergraph construction for each primitive and control-flow.
+
+## Development Tools
+
+### Editors
+
+This month, Louis Gesbert has been busy making the first release of [ocaml-top](http://www.typerex.org/ocaml-top.html), the simple graphical top-level for beginners and students. Together with the web-IDE, this project aims at lowering the entry barrier to OCaml. Ocaml-top features a clean and easy to access interface, with nonetheless advanced features like automatic semantic indentation, error marking, and integrated display of standard library types — using the engines of [ocp-indent](https://github.com/OCamlPro/ocp-indent) and [ocp-index](https://github.com/OCamlPro/ocp-index) of course. The biggest challenge was probably to make everything work as expected on Microsoft Windows, which was required for most beginners and classrooms.
+
+![ocaml-top](assets/img/ocaml_top.png)
+
+The two main issues were:
+
+*   Setup the build environment: there are several versions of OCaml for Windows ; we generally want to avoid any dependency on cygwin on the generated program, but it’s very hard to avoid any need for it in the build chain. The easiest solution at the moment is to “cross-compile” from cygwin using the mingw32 gcc compiler. The hard part is to get all the needed libraries properly setup: this felt a lot like Linux 15 years ago, you can find some binaries but generally not properly configured, and there is no consistent packaging system (or at least you can’t find what you want in it).
+*   Process management: ocaml-top runs the OCaml toplevel as a sub-process, so as not to be inpaired by any problem in the user program. Interacting with that process in a portable way is close to impossible, Windows having no POSIX signals, and read/write operations being very different in terms of blocking, etc. Some obscure C bindings were required to simulate a SIGINT that could tell the ocaml process to stop the current computation and return to the prompt. But at this cost, ocaml-top can be run with any existing external OCaml toplevel.
+
+Not mentioning some gtk/lablgtk bugs that were often OS-specific. After having read [horror stories](http://gallium.inria.fr/%7Escherer/gagallium/the-ocaml-installer-for-windows/) about the most commonly used “Windows installer generator” [NSIS](http://nsis.sourceforge.net/Main_Page), Louis opted for the Microsoft open source solution [WiX](http://wixtoolset.org/) which turned out to be quite clean and efficient, although using a lot of XML. The only point that might be in favor of NSIS is that it can generate the installer from Linux, so it’s much convenient when you cross-compile, which is not the case here ; also worth mentioning, Xen and LVM are really great tools which do save a lot of time when working and testing between two (or more) different OSes.
+
+Always on the editor front, David and Pierrick have been working on a web-IDE for OCaml since the beginning of their internship two months ago. For now, the IDE includes [Ace](http://ace.c9.io/), an editor, plugged with some features specific for OCaml, particularly [ocp-indent](https://github.com/OCamlPro/ocp-indent), made possible by using [js_of_ocaml](http://ocsigen.org/js_of_ocaml/) which compiles bytecode to Javascript. It also includes a basic project manager that uses a server to store files for each user. Authentication is done by using [Mozilla’s Persona](http://www.mozilla.org/en-US/persona/). One particularly nice feature they are working on is _client-side_ bytecode generation: this means users can ask their browser to produce the byte-code of the project they are working on _without any connection to the server_ ! Beware that this is still work-in-progress and the feature is not bug-free for the moment. The project (undocumented for now) is available on [Github](https://github.com/pcouderc/ocp-webedit).
+
+### Tools
+
+Meanwhile, most of my time last month has been spent preparing the next release of OPAM, with the help of Louis Gesbert. This new release will contain a [lot of bug-fixes](https://github.com/OCamlPro/opam/issues?milestone=17&page=1&state=closed) and an improved `opam update` mechanism: it will be much more flexible, faster and more stable than the one present in `1.0`. Few months ago, I had already pushed a [first batch of patches](https://github.com/OCamlPro/opam/pull/597) to the development version, which started to make things look much better. I decided last month to continue improving that feature and make it rock-solid: hence I have started a [regression testing platform for OPAM](https://github.com/samoht/opam-rt) which is still young but already damn useful to stabilize my new [set of patches](https://github.com/OCamlPro/opam/pull/719). `opam-rt` is also written in OCaml: it generates random repositories with random packages, shuffles everything around and checks that OPAM correctly picks-up the changes. In the future this will make it easier to test complex OPAM scenarios and will hopefully lead to a better OPAM.
+
+[ocp-index](https://github.com/OCamlPro/ocp-index) has seen some progress, with lots of rough edges rounded, and much better performance on big `cmi` files (typically module packs, like `Core.Std`). While more advanced functionality is being planned, it is already very helpful, and problems seen in earlier development versions have been fixed. The upcoming release also greatly improves the experience from emacs, and might become the first “stable”. The flow of bugs reported on [ocp-indent](https://github.com/OCamlPro/ocp-index) is drying up, which means the tool is gaining some maturity. Not much visible changes for the past month except for a few bug-fixes, but the library interface has been completely rewritten to offer much more flexibility while being more friendly. This has allowed it to be plugged in the Web-IDE (see above), which being executed in JavaScript has much tighter performance constraints — the indent engine is only re-run where required after changes — ; and in ocaml-top, where it is also used to detect top-level phrase bounds.
+
+## Community
+
+We are proud to be well represented at the [OCaml Developer Workshop 2013](http://ocaml.org/meetings/ocaml/2013/program.html). This year it happens in Boston, in September, co-located with the [Conference of Users of Functional Programming](http://cufp.org/conference/schedule/2013). Both conferences will contains a lot of OCaml-related talks: I am especially excited to hear about [PHP type-inference efforts](http://cufp.org/conference/schedule/2013) at Facebook using OCaml! If you are in the area around the 22/23 and 24 of September and you want to chat about OCamlPro and OCaml, we will be around!
\ No newline at end of file

src/content/blog/news_from_may_and_june.md

+title=News from May and June 
+authors=Çagdas Bozman
+date=2013-07-1
+category=OCamlPro
+tags=Compiler,Devleopment Tools,OCaml,OCamlPro,wxocaml
+
+It is time to give a brief summary of our recent activities. As usual, our contributions were focused on three main objectives: 
+- make the OCaml compiler faster and easier to use;
+- make the OCaml developers more efficient by releasing new development tools and improving editor supports;
+- organize and participate to community events around the language
+
+We are also welcoming four interns who will work with us on these objectives during the summer.
+
+## Compiler updates
+
+Following the ideas he announced in his recent [blog post](http://ocamlpro.com/2013/05/24/optimisations-you-shouldnt-do/), [Pierre Chambart](https://github.com/chambart) has made some progress on his [inlining branch](https://github.com/chambart/ocaml/tree/flambda_experiments). He is currently working on stabilizing and cleaning-up the code for optimization which does not take into account inter-module information.
+
+We also continue to work on our profiling tool and start to separate the different parts of the project. We have [patched](https://github.com/cago/ocaml) the compiler and runtime, for both bytecode and native code, to generate : `.prof` files which contain the id-loc information and allow us to recover the location from the identifiers in the header of the block; and to dump a program heap in a file on demand or to monitor a running program without memory and performance overhead. [Çagdas Bozman](http://cagdas.bozman.fr/) has presented the work he has done so far regarding his PhD to members of the [Bware](http://bware.lri.fr/index.php/Presentation) project and we started to test our prototype on industrial use-cases using the [why3](http://why3.lri.fr/) platform.
+
+On the multi-core front, [Luca Saiu](http://ageinghacker.net/) is continuing his post-doc with [Fabrice le Fessant](http://fabrice.lefessant.net/) and is modifying the OCaml runtime to support parallel programming on multi-core computers. Their version of the “multi-runtime” OCaml provides a message-passing abstraction in which a running OCaml program is “split” into independent OCaml programs, one per thread (if possible running on its separate core) with a separate instance of the runtime library in order to reduce resource contention both at the software and at the hardware level. Luca is now debugging the support for OCaml multi-threading running on top of a multi-context parallel program. A recent presentation covering this work and its challenges is available [online](http://ocamlpro.com/pub/multi-runtime.pdf).
+
+A new intern from [ENS Cachan](http://www.ens-cachan.fr/), [Thomas Blanc](https://github.com/thomasblanc) is working on a whole program analysis system. His internship’s final goal is to provide a good hint of exceptions that may be left uncaught by the program, resulting a failure. It is quite interesting as exceptions are pretty much the part of the program “hard to foresee”. The main difficulty comes from higher-order functions (like `List.iter`). Because of them, a simple local analysis becomes impossible. So the first task is to take the whole program in the form of separated `.cmt` files, [merge](https://github.com/thomasblanc/ocaml-typedtree-mapper) it, and remove every higher-order functions (either by direct inlining if possible or by a very big pattern matching). The merging as already been done through a deep browsing of the compiler’s typedtrees. Thomas is now focusing in reordering the code so that higher-order functions can be safely removed.
+
+Finally, we are helping to prepare the release 4.01.0 of the OCaml compiler: Fabrice has integrated his [frame-pointer](http://ocamlpro.com/2012/08/08/profiling-ocaml-amd64-code-under-linux/) patch, that can be used to profile the performance of OCaml applications using Linux `perf` tool; he has added in `Pervasives` [two application operators](https://github.com/ocaml/ocaml/commit/ace0205b6499ffdae4588cfdd640c45855217a8f) that had been optimized before, but were only available for people who knew about that; he has also added a new environment variable, `OCAMLCOMPPARAM`, that can be used to change how a program is compiled by `ocamlc`/`ocamlopt`, without changing the build system (for example, `OCAMLCOMPPARAM='g=1' make` can be used to compile a project in debug mode without modifying the makefiles).
+
+## Development Tools
+
+Since the initial release of [OPAM](http://opam.ocamlpro.com) in March, we have been kept busy preparing the upcoming `1.1.0` version, which should interface nicely with the forthcoming set of automatic tools which will constitute the first version of the [OCaml Platform](http://www.cl.cam.ac.uk/projects/ocamllabs/tasks/platform.html) that we are helping [OCamlLabs](http://www.cl.cam.ac.uk/projects/ocamllabs/) to deliver. We have constantly been focused on fixing bugs and implementing feature requests (more than [70 issues](https://github.com/OCamlPro/opam/issues?direction=desc&milestone=17&page=1&sort=created&state=closed) have been closed on Github) and we have recently improved the speed and reliability of `opam update`. More good news related to OPAM: The number of packages submitted to [official](http://www.cl.cam.ac.uk/projects/ocamllabs/tasks/platform.html) repository is steadily increasing with around 20 new packages integrated every-months (and much more already existing package upgrades), and the official Debian package should land in [testing](http://ftp-master.debian.org/new/opam_1.0.0-1.html) very soon.
+
+This month, [Louis](http://louis.gesbert.fr/cv.en.html) was still busy improving different tools for ocaml code edition. `ocp-index` and `ocp-indent`, made for the community to improve the general ocaml experience and kindly funded by [Jane Street](http://janestreet.com), have seen some updates:
+
+*   [ocp-index](https://github.com/OCamlPro/ocp-index): the library data access tool which was first presented in [April](http://ocamlpro.com/2013/04/22/april-monthly-report/) has seen some progress, with the ability to locate definitions and resolve type names. It is still not yet considered stable though, expect more from it soon. An early release (0.2.0) is in OPAM.
+*   [ocp-indent](https://github.com/OCamlPro/ocp-indent) the generic ocaml source code indenter, has seen its usual bunch of fixes, along with some new customization options. Also, its [library interface](https://github.com/OCamlPro/ocp-indent/blob/master/src/indentPrinter.mli) has been rewritten, offering much better flexibility and opening the gate to uses like restarting from checkpoints to avoid full reparsing, detecting top-expression boundaries, syntax coloration, etc. We will be releasing 1.3.0 in OPAM very soon.
+
+We are also developing in-house projects aiming at providing a better first experience of OCaml to beginners and students:
+
+*   the new [ocaml-top](https://github.com/OCamlPro/ocaml-top) (previous project name `ocp-edit-simple`) aims to offer a simple, but clean and easy-to-use interface to interact with the ocaml top-level. It is intended mainly for exercises, tutorials and practicals. A release should be coming soon, the Linux version being quite stable while some bugs remain on Windows.
+*   two new interns, [David](http://www.linkedin.com/profile/view?id=3D238971426&locale=3Dfr_FR&tr=k3Dtyah) and [Pierrick](http://www.linkedin.com/profile/view?id=3D65173689), have started working on a [web-IDE](https://github.com/pcouderc/ocp-webedit) for OCaml. As students, they have seen sometimes how difficult it could be to install OCaml on some OSes, or simply configure editors like emacs or vim. To solve these issues, the idea is to use only a web browser-based editor and provide a way to compile a project without having to install anything on your computer. For the editing part, the idea is to use [Ace](http://ace.ajax.org/) and improve it for OCaml, using [ocp-indent](https://github.com/OCamlPro/ocp-indent) for example, which is possible by using [js_of_ocaml](http://ocsigen.org/js_of_ocaml/). The next step will be to glue this editor with both [TryOCaml](http://try.ocamlpro.com/) to execute code, and a cloud computing part, to store projects and files and access them from anywhere.
+
+We are also trying to improve cross-compilation tutorials and tools for developing native iOS application under a Linux system, using the OCaml language. [Souhire](http://fr.linkedin.com/pub/souhire-kenawi/6a/614/54b/), our fourth new intern, is experimenting with that idea and will document how to set up such an environment, from the foundation until the publication on the application store (if it is possible). She is starting to look at how iOS applications (with a native graphical interface) written in C can be cross-compiled on [Linux](http://code.google.com/p/ios-toolchain-based-on-clang-for-linux/wiki/HowTo_en), and how the ones written in OCaml can be cross-compiled on [MacOSX](http://psellos.com/ocaml/).
+
+On the library front, Fabrice has completely rewritten the way his [wxOCaml library](http://www.typerex.org/ocplib-wxOCaml.html) is generated, compared to what was described in a previous [blog post](http://ocamlpro.com/2013/04/02/wxocaml-camlidl-and-class-modules/). It does not share any code anymore with other wxWidgets bindings (wxHaskell or wxEiffel), but directly generates the stubs from a DSL (close to C++) describing the wxWidgets classes. It should make binding more widgets (classes) and more methods for each widget much easier, and also help for maintenance, evolution and compatibility with wxWidgets version. There are now an interesting set of samples in the library, covering many interesting usages.
+
+## Community
+
+We have also been pretty active during the last months to promote the use of OCaml in the free-software and research community: we are actively participating to the upcoming [OCaml 2013](http://ocaml.org/meetings/ocaml/2013/) and [Commercial User of Functional Programming](http://cufp.org/2013cfp) conference which will be help next September in Boston.
+
+While I was visiting [Jane Street](http://janestreet.com/) with [OCamlLabs’s team](http://www.cl.cam.ac.uk/projects/ocamllabs/index.html), I had the pleasure to be invited to give a talk at the [NYC OCaml meetup](http://www.meetup.com/NYC-OCaml/) on OPAM (my slides can be found online [here](http://ocamlpro.com/pub/ny-meetup.pdf)). It was a nice meetup, with more than 20 people, hosted in the great Jane-Street New-York offices.
+
+OCamlPro is still organizing OCaml meetups in Paris, hosted by [IRILL](http://www.irill.org/) and sponsored by [LexiFi](http://www.lexifi.com/) : our last Ocaml Users in PariS (OUPS) meetup was in [May](http://www.meetup.com/ocaml-paris/events/116100692/), there were more than 50 persons ! It was a nice collection of talks, where Esther Baruk spoke about the usage of OCaml at Lexifi, Benoit Vaugon about all the secrets that we always wanted to know about the OCaml bytecode, Frédéric Bour presents us Merlin, the new IDe for VIM, and Gabriel Scherer told us how to better interact with the OCaml core team.
+
+We are now preparing our next [OUPS](http://www.meetup.com/ocaml-paris/events/121412532/) meeting which will take place at IRILL on Tuesday, July 2nd. Emphasis will be on programming in OCaml in different context. Thus, there will be some js_of_ocaml experiences, GPGPU in OCaml and GADTs in practice. There is still many seats available, so do not hesitate to register to the meetup, but if you cannot, this time, videos of the talks (in French) will be available afterwards.
+
+Not really related to OCaml, we also attend the [Teratec 2013 Forum](http://www.teratec.eu/gb/forum/index.html) which brings together a lot of [Scilab](http://www.scilab.org/) users. This is part of the [Richelieu](http://www.richelieu.pro) research project that [Michael](http://www.linkedin.com/profile/view?id=130990583) is working on: his goal is to analyze Scilab code, before just-in-time compilation. It requires a basic type-inference algorithm, but for a language that has not been designed for that ! He is currently struggling with the dynamic aspects of Scilab language. After some work on preprocessing `eval` and `evalstr` functions, he is now focusing on how Scilab programers usually write functions. He is currently using different kinds of analyses on real-world Scilab programs to understand how they are structured.
+
+Finally, we are happy to announce that we finally found the time to release the [sources](https://github.com/OCamlPro/ocaml-cheat-sheets) of our OCaml [cheat-sheets](http://www.typerex.org/cheatsheets.html). Feel free to contribute by sending patches if you are interested to improve them!
\ No newline at end of file