WE OFFER A FREE CONSULTATION
Add file
98

Multitude of Clojure Tools: Huge Advantage or Big Problem?




MULTITUDE OF CLOJURE TOOLS:
HUGE ADVANTAGE OR BIG PROBLEM?

DEVELOPMENT
ALEXANDRA LOZOVYUK
VLADIMIR PAVLYUK

Want to start a project?

We've already talked about why developers choose Clojure and call it the most "elegant and enjoyable" language.

Also, we've gathered insights about the possible reasons for the unpopularity of Clojure. A sparse and undeveloped development toolset was listed among others.

So, how to improve the existing Clojure tool stack? What is happening right now in the community? When will we be satisfied by a fully-loaded Clojure machine and how its technical inspection will be handled? Let's think together!

In this post, we'll talk about the current state of the broad Clojure tooling ecosystem, contrast the approaches and tendencies we see today, including their strengths and weaknesses.

1. Clojure Tooling Landscape: TENDENCIES AND ISSUES

Daniel Higginbotham, the author of "Clojure for the Brave and True", called learning a Clojure a journey through the Four Labyrinths:
The Forest of Tooling
Efficient and friendly programming environment making ideas happen and simplifying dev life.
The Cave of Artifacts
Rules of building, running and distributing your own programs and Clojure's relationship to the JVM.
The Mountain of Language
The mightiest Clojure ecosystem; syntax, semantics, and data structures.


The Cloud Castle of Mindset
Concept of functional programming and philosophy of simplicity that permeates Clojure.


Let's make a stop near the Forest of Tooling today. Clojure development basically requires:

  • IDE
  • REPL
  • dependency and build tool(s)

Instead of using frameworks, Clojure offers a number of powerful libraries that could be put together. It's a really handy approach, comparing, for example, with the JS development universe, where you typically have to learn a number of different tools and learn how they have to be used properly together.

Such a concept provides fast and quite easy tool development. It's such clay in the developer's hands to model everything he(she) wants. Some tools have become common "household names" amongst the devs, others are riding the bench, the third ones were proved unsuitable.
It's easy to become overwhelmed by the sheer number of incomplete tools that can be included in a single project only. That is not the best factor for Clojure's 'ergonomic' and it spoils people's interaction with its ecosystem.
The tooling is confusing as hell initially. <...> On the other hand: there's still innovation going on in that space. Many things in Сlojure are a la carte for good and bad. Bad, because you need time to evaluate the options available, good because you have all those options to choose from.

Jochen Bedersdorfer
CTO at Sematell GmbH
Source: dev.to
The main issue we are considering is a tendency of such a heuristic strategy for reaching immediate short-term goals, that is not guaranteed to be optimal and rational.

What are the risks of such an uncontrolled progression? And how to prevent Clojure tooling entropy?
Do you want to break down myths about Clojure?
Freshcode offers actual research of Clojure challenges and future outlook for free

2. Council of elders. PANDORA's BOX OR SMART TOOL KIT?

Our goal is to become an enjoyable tipster for Clojure newcomers and provide interesting content for all Clojure-lovers. We want to discuss together how Clojure is used in real-world cases, what challenges we've faced and how can we deal with them.

Today we talk about the dilemma of Clojure tooling and the balance between well-established "canonical" Clojure tools and "one-off" things.

How to solve tooling problems like a boss Clojurist? We've asked Sean Corfield and Daniel Slutsky:

  • What do they think of tooling in Clojure: is it an advantage — allowing developers to quickly create tools according to the business needs — or would it make more sense to improve existing tools rather than create a new one?

Clojure titans have generously shared their deep thoughts on the subject.

2.1 Sean Corfield about the importance of using standard staff
Sean Corfield Clojure
I think we have all the tooling now that is needed and it's more about helping people select those tools. The pain points, for people who are just coming to Clojure, are around not knowing how to use the toolset that works with a CLI. There are also guides needed of what set of tools to use with it. I think this is the right place for encouraging people these days.

So on the tooling side, it's not a matter of producing new tooling, it's more a matter of curating a set of tools or making a good set for people.

There are probably things that could be done around that in terms of templates to produce simple, and I emphasize, SIMPLE projects that have all the necessary tools built into them. So maybe creating templates is a good area to look at.

"This is not a tooling question, it's more about the educational resource question."
I often ask: "We have a perfectly good solution here why have you now been creating a competing one?". And the answer is "Yes, it does something but it's not reaching what our team would have done if they would building it".

My position is I like to encourage people to use the standard simple tooling where possible. So I am the advocate of Clojure Spec instead of Schema and others. The main reason for that this is something that is designed by the Cognitect team and fits with the language.

I definitely try to encourage people by saying "Use Clojure CLI, Use Deps, Use Clojure Spec. Use simple mainstream libraries". That's why I still recommend Compojure. It is so much easier to teach people to use it.

If you are producing guided learning then try to stick as much as possible with Cognitect standard staff and with core standard libraries even if some of the others have been very well documented and are objectively better in some way. Because it is a good concept that people need. It's why for example I told people to avoid Mount and guided them to Component. I think that Component has better architecture and it is a simpler library to use.
"I really think that picking a good curated set is a way to solve a tooling problem."

2.2 Daniel Slutsky about the main challenges and solutions
I'll comment from my own perspective, being involved in the Scicloj community, that is building Clojure tools and libraries for data science.

Clojure follows the Lisp tradition of dynamic, playful exploration, and makes it easy to invent new ways of expression. One result of this spirit is indeed a multitude of tools: several excellent IDEs, alongside tools for tasks such as building, testing & deploying projects, data visualization, literate programming, etc. Tools can be customized, and individuals are encouraged to create their own intimate way of interaction with code and data.

All that can be extremely fruitful, but also presents problems, as you hinted:

  1. Maintainability. Will it be possible to keep maintaining all these many tools, where most of them are maintained by very few developers, in a community that is relatively small?
  2. Compatibility. Are the different tools somewhat compatible with each other? This might be important for sharing work across individuals and teams, and for learning resources. For example, if one creates a certain data visualization report in one tool, it may be desirable for others to use that visualization in their own tools, and possibly develop it further with other tools.
  3. Composability. With the multitude of different tools for different needs and uses, can users still have everything they need? Can different tools be combined in harmony in a useful way?
  4. Welcoming beginners. One aspect where the Clojure ecosystem could arguably improve is beginner-friendly resources. To allow newcomers a comfortable entry to the ecosystem, it is important to create experiences that feel familiar and unsurprising. Can this need be in harmony with the spirit of multiple, individualized tools?

The Clojure community seems to develop its ways of coping with these challenges:

(a) Common infrastructure. Tool authors collaborate to create layers they can use across projects. Orchard and REPL-Tooling are some of the famous examples. Another example: for data visualization, both Chlorine and Notespace have adopted parts of the Pink-Gorilla project.

(b) Standardization. There is a tendency to create and adopt standard protocols and data formats, that are used across tools. Quite a few tools use the nREPL protocol for interaction, which is made to be extended for various needs. Data visualization tools such as Pink-Gorilla, Chlorine, and Notespace are adopting common visualization formats so that users can create content on one tool, and then share it for use on another.

(c) Separation of concerns and composition. The Clojure community tends to create tools and libraries that can focus on one task while composing with others for other tasks. Some famous examples are linters such as Clj-Kondo and static analysis tools such as Clojure-lsp, which can be used from several Clojure IDEs. For another example, some data visualization/inspection tools such as Oz, Literate, Pink-Gorilla, Notespace, REBL, Reveal, and Portal, can be used in an editor-agnostic way. This means that users can use their favorite code editor or IDE while enjoying the visualization tool alongside it. Clerk, a new upcoming tool that was just announced yesterday, advocates this approach too. For some of those tools, the composition is done using the above-mentioned nREPL. This hopefully makes the flow of interaction familiar and unsurprising.

(d) Beginner options. That is still a challenge. Hopefully, the fact that most Clojure tools combine well with most IDEs can matter a lot for newcomers, as it allows them to pick a familiar and easy IDE, and still enjoy additional tools without breaking their habits too quickly. Some IDEs, like Calva, put a lot of effort into creating a beginner-friendly environment. So the situation is hopeful.

To conclude, I think the multitude of Clojure tools allows Clojurians to enjoy lots of useful combinations that can be tailored to their own need of playful exploration and creation. The community seems to be coping wisely with the challenges that come with this multitude, but there is still some way to go before the toolset becomes easy, simple, and friendly for newcomers.

3. Freshcode team about Clojure TOOL area

What do you think about the Clojure tooling specifics?

Pros: Clojure enables quite simple and fast tool development.

Cons: many developers ignore the stage of searching already existing relevant tools and skip right to reinventing the wheel.

The problem of Clojure tooling development lies in inefficiency. I am talking about the number of not justified things that were created, just one-time tools for a particular case. This leads to the accumulation of duplicates and valueless tools.

But that's a question for the devs and community, not for the language per se. And, yes, it's cluttering up the tool list, but anyway it's not a problem to find ones you are looking for.
Ilya Dozorenko Clojure Developer Freshcode

What are the 'problem areas' in the Clojure tooling?

New tooling is developed rapidly, also new approaches to existing problems appear. For example, npm integration in ShadowCLJS is much more convenient than cljsjs wrappers, but then many applications still use Figwheel, and the development of new cljsjs libs is stagnating.

On the other hand, development of clj, cljs and some of the core libraries has slowed down. Lack of the core.spec progress has lead to several competing spec libraries (eg Schema, malli).

As for me, I enjoy Cursive's syntax highlight and refactoring abilities but it just can't keep up with all new libraries and approaches (for example, hugsql syntax).

IMO development of Clojure tooling would definitely benefit from coordinated community actions and a long-term roadmap.

1. What is the uniqueness of Clojure tooling?

Clojure allows you to focus fully on the task you want to perform. And I consider it as Clojure's killer feature.

A number of other programming languages require much more integrated and complex knowledge. And, therefore, the development process becomes time-consuming and cumbersome.

For example, working with such enterprise languages as Java, and even JS, you need 1-2 weeks to deal with the project and prepare the environment, wrappers, and some other basic things to start.

Clojure is not one of those. It's a 'thing-in-itself'. Making ideas happen with Clojure is a faster and more autonomic process compared with 'usual' programming languages. The idea turns into MVP here rapidly, as well as MVP turns into a complete product.

2. And what about the growing number of different Clojure tools, both valuable and valueless, as a result of such a fast and easy process? Won't we get such a big attic of unsorted stuff at the output?

To be honest, we've already got it. I've already mentioned why such attics are more
compact in some other languages — building new tools is a much more time-consuming and effort-intensive process compared with Clojure, or, for example, with Python and other scripting languages providing the same benefits.

By the way, it's interesting how the Python community deals with such side effects if are. Actually, there are more generally accepted standards in Python compared with Clojure, and the community is much larger and more skilled given the more considerable age of the language.

So, the attic with unsorted stuff already exists and the temptation to make your own tool is still great. Instead of reading somebody's poor documentation you just create your's own tool, with the same (most often) carelessly written documentation, add it to GitHub, thereby keeping the tradition of untidy tools mixture.

Is that good or bad?
What we have today is definitely bad. Is it reasonable to make tools from scratch? Most probably there is already some ready-made solution you just need to find. If you've been scouring all possible sources, really went all out, and didn't find what you were looking for — here you can start to project your own tool with a clear conscience. But, please, prepare well-structured and clear documentation if you are going to release your code as open-source.

Do you need assistance with Clojure development?
Our team provides consulting and assistance with defining development strategy

4. How to find CLOJURE TOOLS YOU NEED

4.1 Clojure Repositories
The answer is simple and well-known: seek and you will find.
a categorized directory of Clojure libraries and tools, prepared by James Reeves, an author of Compojure, Hiccup, and more.
easy to use community repository for open-source Clojure libraries.
community-powered documentation and examples repository for the Clojure.
P. S.: and don't hesitate to ask Clojure experts to help you. Clojure community is welcoming newcomers, it's such an unspoken rule, or better to say, a free will of its members.

4.2 Other links that may come in handy
Here are also some useful links that may come in handy when looking for Clojure tutorials and how-tos.
Slack Clojure-community with content for newcomers (beginners, clojure, clojurescript) and job search (jobs, remote-jobs)
Clojure subReddit with announcements and discussions of libraries/lectures, etc.
Video lectures & interviews about Clojure by Rich Hickey
We hope you'll get inspired enough to start contributing to some tools (smartly combining strengths of conservatism and reformism :-)) !
If there's one area where Cognitect could have done better it's community building, and engaging more actively Clojurians to help with the growth of the Clojure ecosystem. Here I'm not talking about engaging people to work on Clojure itself, but rather on libraries, frameworks, development tools and all that jazz. :-)

Bozhidar Batsov
Lover of parentheses, Emacs fanatic, Author of RuboCop, CIDER, and more

5. (un)FINAL VERDICT

So, what is the future of Clojure development tools, jokingly called by Bozhidar Batsov "underappreciated workhorses that make our lives hacking with Clojure easier, more fun and more productive"?

Despite a challenging year overall, "2020 was another good year for Clojure and its ecosystem",
according to the State of Clojure 2021 Survey. So we are not panicking about the tooling state in Clojure.

The language is still young and brave and it's going through specific development milestones. Anything that proves itself as an inefficient solution, will be discarded.

    Some discipline required: Clojure is a language that gives us a wide range of options to implement solutions. And it imposes very little ceremony. In essence: with great freedom comes great responsibility.

    Falko Riemenschneider
    Software Engineer, Co-Organizer Clojure User Group Bonn
    Building new tools in some cases really helps to create better products, in the other cases, it breeds 'one-off' things fading into oblivion over time.

    To create great things let's adhere to the principles of:

    • well-organized tooling support
    • consistency
    • retaining the code clarity ("programs must be written for people to read, and only incidentally for machines to execute")
      Please, feel free to write comments below the article if you want to share your point of view.
      Freshcode provides Clojure development services matching your business goals. Do you want to know the benefits of Clojure software development or discuss ways how we can develop the Clojure community together? Please, just fill this contact form and one of our representatives will contact you within one business day.
      What happens after you fill this form?
      • Our representative will contact you within one business day

      • We sign an NDA to keep your ideas confidential

      • You share essential requirements with us

      • Our analysts and developers estimate the budget and timeline
      Shall we discuss your idea?
      THE MOST popular POSTS