would love to hear others opinions

This is not about which programming language any more - but about when will it be possible to develop without programming at all.

I too have this vision about a NLP-Powered Trade Recommendation Engine that can understand news in text form and provide suggestions on impacted instruments. I think we will be there in the not too distant future and I am confident I can make a contribution towards that.

Is the world moving to Machine Leaning based trading ? Any success story of using say Genetic Algo in trading? My bet is that most algo out there are still based on mathematical models to perform statistical abitrage or some momemtum strategies instead of algo that determine market sentiments driven by news-based events. Hope to read up more...

As a layman , I guess HFT is moving to GPU based computing ... Any lang that do that easily would be the next hit.

Thank you my friend, your opinion matters and I would look for the book if I have the chance. It's a big world out there. I am intrigued by talks from people like A. Prof Andrew Ng from Stanford U and breakthroughs in Deep Learning by Google. The tech world is racing to use AI to understand not just spoken words but contextual meanings of sentences to decipher intent and interest. I'm unsure if it's simple pattern recognition or really enabling a machine to think like us ; in trading-terms , the ability to "understand" conversations and predict its net effect and the resulting market trend is tremendous -- it's like being able to "eavesdrop" tons of market conversations (via twitter, chatrooms, headline news etc.) and guess where's the crowd heading before the market opens and adapting the trade position along the way .... the implication is huge and fascinating (if it can be accurately modeled). Anyway, thanks for this interesting thread.

Al is told what to do by a human :-)

There is the issue, that financial trading, and especially strategy development is "conceptual" - you cannot lean a computer to be "conceptual". Innovation is conceptual, combine two different technologies into something new - you cannot learn a computer to innovate.

If you feed it raw data (like OHLC) it's not going to find anything. You have to transform the data, and create calculated fields for it in order to allow it to find things.

Agree Muhammad, a human with a trading idea has to program the computerized system to go through a series of predefined steps - often a sort of "optimization". In my opinion very still far from artificial intelligence, alien talk, self learning computers. But some day it might be possible to have computers at a level where we can call it artificial intelligence.

I use Java for actual trading: speed, flexibility of the oo, and a decent but a bit convoluted API for IB.

Postgres for big data repositories, with some plpgsql for backtesting.

R for analysis, backtesting, trading strategy design, and (cool!) graphs. Amazing for rapid prototyping, but i'm just starting to learn this one.

All of them properly connected, of course. And, btw, all in macintoshes.

Pablo, you say you use Postgres for big data, how much data do you receive during a live session. I receive 2GB during the 6.5 hours of trading, this isn't large in today's standards, but when you are updating aggregates during the live session and looking for targets you have to worry about latency of calculations and updates.

For now it seems to me most of the work to be done by trading algorithms is mathematical, as the calculations and the data grow functional programming becomes more appealing.

I developed some of my code in C# and some in R. I knew the "standard" for programming was OO programming, but my inability to identify pieces of code to encapsulate in classes bothered me. In the back of my head I continuously felt I was doing something wrong. But still it felt more natural to have the data separately in a common "container" available to all functions, and not contained inside objects. Then in the last few months my R programming gotten better, and I learned about functional programming. This made me realize why I was having such a hard time thinking in terms of objects when most of the work consisted of functions and calculations. Overhead needed to be kept at minimum, and code efficiency at maximum.

Hi all. I trade for myself and MQL4 fits perfectly my needs. However, to be quite honest, I should have developed an abstraction layer above MQL4 to simplify and to automatize the access to trading functionalities. So I'm focus on algorithms and gain productivity.

Every time I visit this group, I cant help being puzzled by the complete lack of happy developers posting their success stories - and there are 125.075 members.

I see the same handful of people plus some newcomers writing about what they plan to do - take care not to spoil your life on an Alchemist project, use your time on the family and friends instead.

Søren, because this isn't a one man job. I tried in the past to collaborate with programmers to develop something but I couldn't find any. I had to learn to program in order to develop my system. I don't think many people would do that, it's very tough to have the discipline to learn so much in a completely new field. I did it because I couldn't trade manually, the choice was either stop trading or move to automation.

I met many that were enthusiastic in the beginning, but they weren't willing to do any programming without me telling them everything I know about trading from A to Z and without proving to them that what I want to do works. I couldn't do that because if I were to do so, I will never see them again. There are businesses that would develop the code for you, but their arrangements are even worse. So I had to do it myself.

Most of the people here are either IT experts or traders. Very few are great at both, and I mean very very few.

I am anticipating some would say why didn't you pay someone to do it. The problem is as a manual trader, you do things based on what you see on your screen, they are in your mind but it's very hard to list them all. I needed the luxury to be able to look at the code and tweak it from time to time add to it, replace some of it. I did contract out pieces of my code, but I needed to be able to put the pieces together so I had to learn to program.

Machine learning and AI approaches, in general, will never be able to take advantage of unexpected, unclassified information (e.g., unexpected earnings reports, government collapse, etc.). However, the analysis of sentiment based news analytics strongly demonstrates the amount of "pre-release" information trading in the markets. This, along with vast amounts of market price manipulation will always be problematic for AI models that exist today. Combine that with a healthy dose of irrational human "intuition" and its no wonder many technical indicator-based strategies are not performing as they once did. Rather, an approach that measures what conditions ARE, and reacts to that (rather than predicting future market conditions) seems like a logical approach. You should give it a try. ;-)

This was my take on the question: http://jonathankinlay.com/index.php/2015/02/comparison-programming-languages/

Part 7: Swift (parallel scripting language), SYMPL, SyncCharts, SystemVerilog, T, TACL, TACPOL, TADS, TAL, Tcl, Tea, TECO, TELCOMP, TeX, TEX, TIE, Timber, TMG, compiler-compiler, Tom, TOM, Topspeed, TPU, Trac, TTM, T-SQL, TTCN, Turing, TUTOR, TXL, TypeScript, Turbo C++, Ubercode, UCSD Pascal, Umple, Unicon, Uniface, UNITY, Unix shell, UnrealScript, Vala, VBA, VBScript, Verilog, VHDL, Visual Basic, Visual Basic .NET, Visual DataFlex, Visual DialogScript, Visual Fortran, Visual FoxPro, Visual J++, Visual J#, Visual Objects, Visual Prolog, VSXu, Vvvv, WATFIV, WATFOR, WebDNA, WebQL, Windows PowerShell, Winbatch, Wolfram, Wyvern, X++, X#, X10, XBL, XC (exploits XMOS architecture), xHarbour, XL, Xojo, XOTcl, XPL, XPL0, XQuery, XSB, XSLT - See XPath, Yorick, YQL, Z notation, Zeno, ZOPL, ZPL. Best Regards, Valerii

Some statistics: I) languages named by single English letter are B, C, D, E, F, G, J, K, L, Q, Q, R, S, T, Z - these are 15 of 691 (Q and Q is not a typo ... there are two Q languages); II) "language++" A++, C++, J++, Karel++, R++, Turbo C++, Visual J++, X++ but ... there is C-- from Simon Peyton Jones & Norman Ramsey; III) ISO/IEC Standard is available for: Ada 8652 1980, APL 8485 1964, C 9899 1972, C++ 14882 1985, C# 23270 2000, COBOL 1989 1959, Eiffel 25436 1986, Fortran 1539 1957, FULL BASIC 10279 1964, ISLisp 13816 1958/1997, Modula-2 10514 1978, Pascal 7185 & 10206 1970, PL/I 6160 & 6522 1964, Prolog 13211 1972, Ruby 30170 1995, Z 13568 1977. Best Regards, Valerii

Jonathan, great great answer.

Valerii, really?

Revolution R (R RE) is integrated inside MS SQL server starting 2016, the following article claims R RE achieved 300x speed compared to conventional R. And with R RE integrated in SQL I think much of the speed issues are going to be gone. See this article: http://www.r-bloggers.com/benchmarking-results-compare-speed-of-revolution-r-enterprise-and-legacy-sas/

Muhammad, "really" means, first of all, that all languages in the list are real. I practice a few of them and some for very long time. My recommendation would be biased. Best Regards, Valerii

Vlaerii - pure programming is today a complete wast of time. How do you think solutions such a Skype are developed - by pure programming from zero and up ?

I find banks always insist on C and its derivatives, i.e. C++ and C#. Python is laying second, while Java and Java script are third.

So, object oriented languages are the base, and this is surely for ULL.

When talking about servers and database, they say linux and MySQL , respectively.

IMVHO, I am trying to find a coder who understands all of the above to automate my rules as well as backtest them and makes statistics with R or Matlab "another comparison needed", and I didn't find yet.

If you really rally want to use programming despite visual tools have now taken over 98% of the development - I am puzzled with no one have mentioned MathLab, the industry snatdard. I gather it is causemany dont know what they are doing and what they suggest. MathLab kinda defines which programming language you use.

Before one put all kind of questions, which are ABC questions, I suggest instead to make a research on your own of how the professionals do this. A hint is to check the professional seminars, and which platforms and tools they use in their seminars to tech professionals new methods within strategy development.

Every language is good, if works correctly for you private use. Next if you want sell it you have to make algo in a language your client use. I remember one day when one person laughed from me I wanted made stat arb in MQL4. I did it, many stat arbs in this language and have been working very well. Of course these stat arb are not for HFT. Sometime broker doesn't offer API, sometime broker wants to see your algo (how it works) - for example Lmax ( I talked with Lmax about API and algo in Java). I think the main question is: what you want to reach via your algo instead of which language is better.

In general when we say algo. trading, it implies lots of math calculation and highly efficient code. Now, if you only look at the S&P futures to trade any language would work. But, if you track many instruments, like a 100 or more, and have to do many calculations some languages are be better than others. C and C++ are probably the baseline but development with these languages is expensive, this is why we are having this conversation.

You can't do these kinds of jobs without language agility. Once you know the fundamentals though, you won't be so hung up on language.

Charles, wouldn't you want to know though beforehand base R for example is 400x slower than C, wouldn't be beneficial to know Revolution R can be as much as 300x faster than base R? Wouldn't that play a roll during your planning phase?

I would not call C++ a "derivative of C". At the beginning (1979), "C with classes", C++ since 1983, also got some ideas from Simula. It supports C. It allows varieties of programming styles: functional, object-based, object-oriented, generic, metaprogramming. If we can speak about the nearby trend of C++, then generic and metaprogramming are getting strength due to Strosutrup's templates innovations and Stepanov's revolutionary algorithmic ideas. It difficult to think off about a modern efficient alternative language supporting both. Best Regards, Valerii

"Søren Lanng Vlaerii - pure programming is today a complete wast of time. How do you think solutions such a Skype are developed ...". I think this rocket is made from lumber programmed in a forest of components programmed using high and not always high efficient languages. Best Regards, Valerii

The quality of the programming dominates the language used. For HFT, C++ is needed for speed, but this is a rare exception where the choice of language is important.

I think choice of programming language is very important - C++ the most important language, cause it is fast, it is very standard - and very importantly C code can be ported to almost anything - DLL, CGI, PHP, Javascrips (as these languages are very similar), VisualStudio, Borland. Most professional development environments use C++/C/C#.

Keep the code very simple, avoid platform specific function calls, if a platform specific call is made, put the calls in a black box function one place - in case you need to port your code in the future (very likely).

But before deciding on programming language, I think first to make a research, lay a plan as to the most easy manner to develop and maintain strategies - many work headless without any planning, and usually end up with nothing achieved after 10 years.

A lot of things are used in a trading firm. A blackbox trading firm will use a combination of Java, C++/CUDA, and some languages that are good for mathematical analysis like Matlab, Python, or R. But a lot of the best trading firms have their own in-house programming language.

At MIT, we emphasize the learning of Matlab, Python, and R for data analysis.

Also, I can compare Java versus C++. For the exact same program written in C++ and Java, the Java program generally has a longer run-time. Thus, C++ is more useful for executing trading orders at a faster speed. There are some drawbacks however. This is why a lot of HFT firms have their own coding language, in-house.

"Søren Lanng: I am puzzled with no one have mentioned MathLab, the industry snatdard." There are two incorrect and one questionable thing in one sentence: 1) the language is MATLAB (not MathLab) - no 'h', FYI, this is a registered trademark with R symbol; 2) I did mention it my Part 4 as a language to choose: " ... MASM Microsoft Assembly x86, Mathematica, MATLAB, Maxima (see also Macsyma) ...", 3) it is used with other tools such as Mathematica, Maple, S-Plus, etc. Best Regards, Valerii

Mathlab or matlab, dont use either of them - have developed my own development platform which has this build in natively supported, all programming has been replaced with visual methods - big data ( 8-16GB) in a development and trading platform holding and synchronizing all data, cross market aware, edit and backtest a strategy in one single event by a mouse click < 500ms. Optimization using up to 16 full strategies as input, 20.000 tests per second. By having this I can test thousands of ideas in a afternoon, implement any idea I want - if I need some custom criteria I can add them in a special module by programming. TIME is the key today, to explore new opportunities today and not in months or years. To focus on the idea and not on programming and platform development/maintenance. This platform will be set out free in the market for users to use as a "mock-up" tool to speed up their development, or use it in it complete form - as they please.

As to which database to use - the answer is none. In the years of development attempting to reinvent the wheel, you can expect to spend half on fighting with the database. Use the OS filesystem instead - use strings only, fixed record and row lengths. The file path is the search path.

If you are smart, you have one central black box function where you perform all the read/write/search.

Can I play devil's advocate please?

Java!

It runs quite happily on a headless Raspberry Pi, and pretty much anything else under the Sun, according to the Oracle at least.

Jim, not sure if there is a standard by which IT people measure calculation intensity, but how would you describe the systems you tested on a scale of 1 to 10 with one being least intense characterized by OHLC one minute data for under 10 instruments, and 10 being tick data for 500 symbols or more with continuous aggregations and calculations update with complex relationships between the various data symbols.

our company used C, C++

LMAX assured me that their entire MTF is written in Java:

http://trading-gurus.com/is-quickfix-j-broken/

Is that intense enough for you?

As to our experience QuickFix is full of bugs and rather unreliable - when integrating to a broker, we can immediately see when side is based on QuickFix. Many develop their own fix engine in the conclusion QuickFix is too unreliable.

A typical problem for QuickFix is hanging connections.

Apart, you do not write anything mission critical in Java.

@ Jim, watched the video it's interesting! Can't claim understood everything in it, but seams they developed their own "advanced" way of dealing with communication bottlenecks. During the 25th minute he says something like "writing code in Java or C# is like a decade old way of doing things". I am not contesting that one can write a fairly advanced system with Java, but it seams you would need to be aided by something similar to the LMAX technology. Also the example of serial processing vs. parallel processing was about a simple operation. In a complex system, prices get updated, then "aggregaters" update the aggregates, then targets are compared, cross-relationships are assessed, and critical points are also assessed. There is lots of back and forth, and all must be done with minimal latency. Maybe this continuous back and forth is where you need C++.

correct me if I was wrong, the LMAX was about the communication phase of the system, but the most demanding thing is going to be the calculations done internally. The data received is the raw data, but a good system has lots of calculated fields made out of the comparisons and aggregations. It certainly isn't about sending buy/sell orders if a certain raw data value is achieved.

I am not an expert on this I am having this discussion to learn from the opinions that everyone shares here, so thanks Jim for the link, it was very informative.

i'm surprised no one mentioned Python. It takes 3 times less time to code smth in Python vs Java or C++

Python certainly has its place here, too. However, it really depends which part of the overall system should carry out which task and which language to apply specifically there. Speedwise e.g. Python is not always dramatically slower compared to e.g. C++(https://benchmarksgame.alioth.debian.org/u64q/python.html), but in very many cases it is. Having programmed many years in Ada, Clipper, FoxPro, C, XoJo, I personally chose FreeBasic (for ease of coding, and general purpose character, + homegrown special purpose libs) with its C backend for fast results with speedy execution code. In case much backtesting or analysis of huge piles of data becomes necessary, you might want to apply methods and tools from the BI domain to pre-process the data and use e.g. C for the algo itself. General advice: if a part of a systems seems to be a (performance) bottleneck try to cope in the overall algo first, then check the architecture and finally try to speed up with implementation :)

Surely a good way to go is use a high performance language for execution eg C++ and then make the API/ Rest interface language agnostic, I can see arguments for using python, R, C#, Java and C++ or some functional language like Haskell too.

Hmmm. LinkedIn broke my last link, and won't let me edit/delete the post.

Hunt around a bit and I expect you'll find the QCon 2015 stuff.

c or cpp , python is way slower for algo

but besides algo the connection matters , lower latency is better for such trade

When it comes to developing an automated trading system, it comes down to what choices you have and what you need to achieve. As a retail trader my gateway in IB and my choices are java, C# or C++. For me it's a matter of I like best and what will get me there fastest. At the moment I'm using Java for Mac and C# for Windows.

Historically all the top Prop Firms have used C++ Unix because it is fast clean code. These shops now are hiring Traders that can also code R. My firm has placed many elite software engineers and Traders coming from top firms, 90% fit into this mold.