Connections Between Financial Signal Processing, Entropy, and Superior Investment Returns ?

 

Summary

 

Have you ever lost money in investment ? If you have, do you know what Newton’s thoughts on investment loss were ? And how did Newton’s reflections on investment hint the beginning of the application of physics and science concepts to investment practices ? Do you know that the mad behavior of people in investment bubbles may be partially understood with science concepts, such as entropy and phase transitions ?

 

This article touches upon questions. From Newton’s reflection on investment bubbles, this article introduces  connections between  physics, communication technologies and investment; focusing particularly on the ways in which physics concepts influence modern communication technologies, financial theories and investment industries. Two major connections are particularly worthy of our attention because of their importance in investment: the connection between entropy and quantitative investment, and the connection between heat diffusion equation and the financial crisis. This article also points out how prominent scholars like Isaac Newton, Ludwig Boltzman, Claude Shannon,  Jim Simons and many others  have  shaped the investment fields. The fields still have lots of challenging and important open questions to be explored. Among these open questions, this article raise a question if the formation of a financial bubble can be modeled similar to critical phenomena in many body physics.

 

1. Isaac Newton’s Investment

 

When discussing physics, starting with Newton could be the right thing to do.  Newton has laid the foundations of many disciplines in modern science. By developing calculus, classical mechanics and the theory of gravity, he was able to calculate the movement of the planets.  For the purpose of this paper, it is also worthy to mention that Newton held the position of Master of the British Royal Mint from 1699 until his death in 1727. 

 

Interestingly, during his lifetime Newton experienced one of the most famous investment bubbles in history.  There was a rumor that the South Sea Company had discovered large oil fields. After the news spread, people went crazy and wanted to invest money in the company. Many people camped outside the investment agents’ offices with bags of money in their hands, trying to take part in the game. It became as crazy as it could get.

 

Later, after the dust had settled, many people lost their money, as it usually happens after investment bubbles.  Converting to today’s value, Newton lost about one million and six hundred thousand British pounds in that bubble (about $2 million).  If you have friends who have lost money during a tech bubble or a housing bubble, you can comfort them by telling them:”You may have done better than Newton.” 

 

Of his loss, Newton said:  “I can calculate the movement of the stars, but not the madness of men"[1].  This statement may sound a bit low-key, but in fact it demonstrates Newton’s wisdom.   He felt that in order to describe human madness, there was a need for other concept that had to be found beyond the realm of classical dynamics. The first concept to be discovered was entropy.

 

 

2. Boltzmann Entropy

 

In the 19th century, thermodynamics, a scientific discipline rooted in the research of heat diffusion, became a mature discipline based on the theory of molecular movement. The pinnacle of this achievement was the statistical explanation of entropy, proposed by Ludwig Boltzmann, J. Willard Gibbs, Rudolf Clausius and others.

 

In basic terms, entropy is the measure of how organized or disorganized a system is. For example, a glass cup which is broken to pieces has greater entropy than an unbroken one, because the broken glass cup is more disorganized.

 

In thermodynamics, the definition of entropy is the following:

                                                                                                                (1)

where k is Boltzmann constant,  is the probability of the “microstates”, and 。The more disorganized a system is, the more microstates it can have, resulting in higher entropy. 

 

 

Entropy is a fundamental concept for describing the “disorganized level” of many body systems, but it has not been applied to investment practices directly from thermodynamics. Perhaps, if Newton had been familiar with the concept of entropy, history might have been different. Boltzmann, however, was probably not involved in investments like Newton; therefore, he didn’t try to apply his theory of entropy to investing. Eventually, the concepts of entropy entered the financial world through the development of modern communication technologies.

 

3. Modern Communication Technologies and Shannon Entropy.

 

During World War II, the US army sponsored communication technology research. This led to advanced radar and microwave technologies, and the Allies benefited greatly from this research. After the war, the army continued sponsoring advanced research in the area of communication. Claude Shannon (1916 – 2001), who worked for the Bell Laboratories, and later studied and worked at MIT, made major breakthroughs in this area, and laid the foundations of the modern communication theory. This happened at the end of the 1940’s [2]. A few of his important contributions are the following:

 

Conceptually, a communication system can be modeled as two ends and a channel. One end is a transmitter, and the other end is a receiver. The channel is the way between the transmitter and the receiver. Usually, a channel has noise, and signals will be attenuated or distorted on their way through the channel.  For practical communication scenarios, the transmitter continually sends out signals and the receiver gets many signals. The signals received are often quite different from the ones transmitted because of signal loss and distortion in the channel. Communication technologies try to recover the original signals from the received ones, and transmit as many signals as possible at any given time through a particular channel.

 

 

 

 

 


Figure 1, Conceptual Diagram of a Communication System

 

Consider DSL, the most deployed broadband access technology. DSL has two ends. One end is the DSL Modem, which is usually located in the consumer’s home or office. The other end is a DSLAM, managed by telecom service providers. In this case, the channel is the telephone wires connecting the modem and the DSLAM. The Internet speed used in everyday conversation often actually refers to the data rate of DSL that a user can get.

 

Another example is wireless services where a mobile phone is one end; a base station is the other end. The space between the mobile phone and the base station is the channel.  If possible, people usually want to get higher transmission data rate regardless of the wire line or the wireless communication.

 

Shannon’s major contribution to communications is that he found the theoretical upper limit of the data rate of a communication system.  In communication terminology, the theoretical upper limit of the data rate is called capacity. Shannon found that the capacity of a communication channel is [3]:

 

                                                                                                    (2)                                                          

 

where C is the capacity,  is the bandwidth of the frequency bands to be used in the communication,  is the signal strength, is the noise strength, is the signal noise ratio SNR (In general, however SNR depends on the frequency, so the right side of the formula needs to be changed to an integral.)

                                               

The above formula shows that the maximum data rate of a communication system depends on the frequency band and the signal noise ratio. This formula is fundamental to communication technologies because the design goal of a communication system is often to optimize various design parameters in order to make the system run as close as possible to the capacity in the formula (2).

 

For example, before DSL technologies were invented, the way to transfer data over phone lines was through voice band modem. This technology used only a very narrow frequency band, with B = 4 KHz (Kilo Hertz), and the maximum data rate that could be achieved was 56 kilobits/second. Among the early DSL technologies, ADSL1 used a bandwidth of B=1.1 MHz (Mega Hertz), and it could achieve a data rate of a few Megabits/second. ADSL2+ used B= 2.2 MHz bandwidth, and it could deliver up to 24 Megabits/second data rate. Among the subsequent DSL technologies, VDSL2 used B=17 MHz bandwidth, and it could deliver up to 30 to 100 Megabits/second data rate (though the practical data rate that a specific user can get depends on many factors, such as the distance between a DSL modem and a DSLAM).

 

The above paragraph may suggest that it is quite easy to increase the data rate of a communication system by broadening its bandwidth. In practice, however, there are many difficult technical problems that have to be solved. One group alone, Stanford University professor John Cioffi and his students spent nearly 10 years on early DSL technologies research, and it has taken almost 20 years of development from early DSL technologies to evolve to today’s VDSL2+.

 

Another way to increase the data rate of a system is to increase its signal noise ratio (). Often, there are restrictions on signal strength, and it cannot be increased without limit. With a given signal strength, various technologies have been developed to reduce noise and thus to increase the data rate. Using DSL technologies as an example, after VDSL2+, a noise cancellation technology based on “cooperation signal processing” was proposed [4, 5], often called VectorDSL or MIMO-DSL. Such technologies, proposed by ASSIA and other companies, can further increase the data rate over telephone lines up to a few hundred megabits/second, which is comparable to the data rate offered by fiber to home or fiber to curb technologies (such as PON), but only with a fraction of the cost.  A special case of MIMO-DSL: phantom mode communication recently have been demonstrated in lab..

 

Such dependence of the data rate on the frequency band and signal noise ratio applies to wireless communication as well. Different wireless technologies often use different frequency bands and have a different signal noise ratio.  For wireless communication, frequency band is one kind of scarcity resource. Telecom carriers, service providers (and Google) often fight fiercely for getting the right to a frequency band.

 

The above illustration of communication technologies is meant to show the importance of Shannon’s contributions to modern communication technologies and its impact on our daily life.  How did Shannon discover his theorem? It turns out that Shannon discovered his famous capacity theorem by studying Shannon entropy.  Following is an example of Shannon entropy.

 

An example of Shannon entropy: a reporter went to South Africa to watch World Cup final  between the Team Netherland and the Team Spain. In order to send out the game results as soon as the game was over, the reporter prepared two short messages ahead of time: “Netherland won”, “Spain won”. He planned to pick one of the two messages at the end of the game and send it out. In this case, Shannon asked “what would be the amount of information contained in the message the reporter would send ?”. Before the game, the outcome of the game is unknown, so we are not able to know which message will be sent out. We assume that both outcomes have a certain probability to happen, and we can study the “average amount of information” contained in the messages to be sent. 

 

Let the probability of the two possible outcomes be：,, 

 

For this example, Shannon found that H in the following formula can describe the “average amount of information” in each message [3,6]. 

 

                                                                                (3)

 

Here, H is called Shannon entropy or information entropy.

                                                                                 

For a general communication system, the messages (symbols) to be sent will be more than two. Let’s assume that the probability that each message will be sent is , and then Shannon entropy of this system will be:

 

                                                                                                   (4)

 

Comparing formula (1) and (4), it is clear that Boltzmann entropy and Shannon entropy are very similar. The similarity is not only in the mathematical formula, but also in the physical meaning. The more disorganized a system is, the bigger is its Boltzmann entropy. The more disorganized a system is, the more information it contains, and the bigger is its Shannon entropy. In other words, both Boltzmann entropy and Shannon entropy describe the “degree of disorganization” of a system. From this viewpoint, the second principle of thermodynamics, the principle of entropy increase, can be understood as the principle of information increase, which does not sound as bad as the order of disorganization increases.

 

Why does a more disorganized system contain more information? This might sound a bit contradictory to intuition. Following is an example that could possibly offer some explanation. We want to color a paper grid and we want to do it in one of two ways. One way would be to use the same color, such as blue, for all the squares.  Another way would be to use different colors randomly for each of the squares. Then, intuitively, the paper colored in the second way will be more “disorganized” than the paper colored in the first way. Which paper will contain more information? The paper colored in the first way contains very little information, because, in order to describe it, we should only say that all the squares are blue. In order to describe the paper colored in the second way, however, we have to describe the specific color of each square. This example also illustrates the foundation of digital compression technologies, which has been used widely in image and video compression and transmission. The maximum degree that a system can be compressed is determined by the amount of information it contains described by its Shannon entropy.

 

Shannon not only discovered the capacity in a communication system (2), but also defined the unit of information called the ‘bit.’ Because of the wide use of communication and computer technologies in everyday life, the notion of ‘bit’ has become very common in our everyday vocabulary. You may not even realize that you often use this word. For example, you may say, “the DSL speed of my Internet service is 24 Mega,” “my wireless data rate is 1.5 Mega,” or “I bought a 20G external disk.” Here, 24 Mega is  bit/second, 1.5 Mega is  bit/second, 20G means that the storage capacity of the external disk is  bytes, and as one byte contains 8 bits, the storage capacity of this disk is  bits. You may notice that in the above example, a ‘bit’ is the unit of communication as well as the unit of computer storage. In fact, it is the ‘bit’ that establishes the connections between modern communication technologies and computer technologies, unifying these two very important fields.

 

How is a bit defined? In our example, if the team Netherland and the team Spain have an equal chance to win,, then formula (3) will give = 1 bit. In this case, the amount of information contained in the message to be sent by the reporter is 1 bit.  Another example of 1 bit in the following physics example: 

 

Entropy of a free electron in vacuum:

 

The electron has only two possible microstates, spin up or spin down. The probability that these two states might occur is the same;. According to (3), the Shannon entropy of the system is 1 bit; According to (1), Boltzmann entropy for it is =  . Because both Shannon entropy and Boltzmann entropy describe “the degree of disorganization,” they are the same.

 

* bit =                                                                (5)

 

The above formula is interesting because the unit on the left side of the equation represents the amount of information, while the units on the right side represent energy (Joule) and temperature (Kelvin). Because of the impact of Internet, more people believe that information is as real as energy, but how do we interpret the connection between information, energy and temperature in an equation like (5)? Boltzmann constant is one of three fundamental physical constants. The other two, the speed of light and Plank’s constant, have revolutionized physics.  Can Boltzmann constant and equation (3) lead us to another fundamental breakthrough ?

 

Besides making fundamental contributions to communication and computer technologies, Shannon had a great interest in stock investment.

 

4. Shannon and Investment

 

Considering stock investment, the approach of a person like Shannon would naturally be different from the approach of ordinary people, and in fact, he did try to apply scientific concepts to his stock investment. However, Shannon did not publish any papers on his research on stock investment, he only gave two talks on the subject at MIT. From these talks, we can understand that Shannon and his students have tried to develop algorithms and computer software for stock investments as early as the 1970’s. For this purpose, Shannon studied a strategy called the Constant Re-balance Method[3, 7].  A simple form of this strategy is that for a given stock portfolio, for example, an index fund, an ETF or a mutual fund, and if an investor continually “sells high and buys low”,  that is, he sells the stock which performs well and buys more of those that perform poorly in the portfolio, such an action is called “rebalance”. Shannon found that this strategy would produce good investment returns in the long range during the time period Shannon studied the market. How often should the investor “sell high and buy low”? This is a parameter which can be adjusted. If rebalance is done often, it will become one of high frequency trading methods. Besides stock price, the Constant Re-balance Method can be designed to do re-balance based on other investment requirements.

 

Shannon himself did not use the above methods in his personal investments, because he found that the Constant Re-balance involves high transaction fee. Furthermore, in the 1970’s, computer technologies and networking technologies were not mature yet, which made it difficult to apply this method in practice. It is claimed, however, that some quantitative hedge funds did integrate this method into their investment methods during the 1980’s and 1990’s.

 

Nevertheless, the return on Shannon’s personal investment was successful. As he had close connections with the technology industry through students and colleagues, he chose to invest in the companies of  his friends (In today’s terms, “invested in a team”). The companies in his portfolio included successful high tech companies, such as HP and Teradyne. During the same time span, his investment return managed to beat Warren Buffett’s return, even though this might have been well below Shannon’s own expectations.

 

Even though Shannon himself did not apply communication theories to his investments (Constant Re-balance Method is related to Shannon entropy [6, 7]), he influenced many people who did apply communication theories to investment. Let’s start by introducing of J. L. Kelly, Jr, Edward O. Thorp, Elwyn Berlekamp and Thomas Cover.

 

5. Kelly and the Capital Allocation Problem in Investment

 

J. L. Kelly was Shannon’s friend and colleague. He studied communication theory at Bell laboratories in the 1950’s, and found that a communication system can also be studied as a “continual gambling system”. It was this study that led him to discover Kelly Criterion[7, 8].

 

A simple example of Kelly Criterion[9, 10]: Assume there is a gambling (or an investment) opportunity, you may bet (or invest) continually. If the chance that you win is p=0.6, the chance that you lose is 1-p = 0.4. And if you win, the money you invest will be doubled, whereas if you lose, the money you invest will be lost (Double or Nothing). Then, how much money should you invest each time in order to get the best return in the long term? Clearly, it is not wise to put all your money at once, because if you make a wrong bet, you will not have a chance to recover your loss.

 

The correct answer is: 2p-1 = 0.2                                                                            (6)

                                                                                                         

Each time you shall use 20% of the money for betting. Furthermore, Kelly found that you can expect to double your money after 36 bets (in general, if the chance to win is p, the answer to this question is 2p -1,  if p > 0.5.  If p < 0.5, you should not participate in the game).

 

Clearly, this is a conclusion related to investment, but Kelly discovered it by researching communication capacity formula (2). You may realize that Kelly’s Criterion is useful for fund managers in order to allocate capitals; if a fund manager has an estimation of  the probability of winning based on models and historical data, then the Kelly criterion tells him how much money the fund manager should invest each time.

 

It is not clear whether Kelly has ever used his methods in his own investment, but a few people influenced by Shannon made an impact on the science of investment.

 

6. Thomas Cover, Edward O. Thorp and Elwyn Berlekamp

 

Stanford University professor Thomas Cover from has studied  the application of Shannon entropy to investment.  He proposed an interesting and somewhat controversial theory, the Universal Portfolio Theory[6], and he proved theoretically that Universal Portfolio can lead to an optimal investment strategy.  In practice, however, he did not specify how to construct a universal portfolio.

 

The Universal Portfolio Theory is well recognized in the community of information theory researchers, and Cover was given the highest award in information theory, the Shannon Prize. His theory, however, was criticized by a famous economist, Nobel Laureate Paul Samuelson (1915 – 2009). Cover tried to apply his theory to practice and started a hedge fund, there is not much information about this fund.

 

Edward Thorp was one of the early adopters of quantitative investment. He was a professor at UC Irwin, and was running several quantitative hedge funds [7,9]. He used Kelly criterion in his investments. He also published many articles on quantitative investment, an uncommon move, as most hedge fund managers do not usually speak about their investment strategies, even years after they have left the funds.

 

Elwyn Berlekamp is a professor emeritus of mathematics at UC Berkeley. He has broad research interests and has made great contributions in the area of communication theory, cryptography, and mathematics. For example, he invented a decoding method for the widely used Reed-Solomon encoder, which is used in CD players, DSL modem and many other fields. He started a cryptography company and led it to an initial public offer. His latest research area was the study of the mathematical theory of the end game of Go. (Go, or “Weiqi” in Chinese, is a popular board game in China, Japan, Korea and Taiwan, as well as parts of the US.*Note by editor: the rules of Go are considerably simpler than the rules of chess; however, Go is much more difficult for computers to play well.) In January 2009, the author met Professor Berlekamp at a Go tournament in San Francisco, and asked him whether his Go research actually helped him with the game. He answered, “By the end game, I am already behind too much.”  (A side note: In the late 1940’s Turing and Shannon studied how to play chess with a computer, and their study was very important to theoretical computer science. Berlekamp himself studied computer chess problems in the 1960’s. )

 

An additional reason why we introduce professor Berlekamp is his connections with the famous hedge fund company, Renaissance Technologies Inc.

 

7. Renaissance Technologies Inc. and James Simon

 

Renaissance Technologies Inc. is one of the most successful hedge fund management companies in the world.  Simon started the Renaissance Technologies Inc. with about $6 million in 1982. The most successful fund of the company is Medallion fund . After years of excellent investment returns of the Medallion fund, his current personal assets amount to approximately 8.5 billion dollars according to Forbes (March 2010 data). The Medallion fund was actually started by Axcom Trading Advisors, and Berlekamp had played a very important role in Axcom. He was its biggest shareholder, as well as its president and CEO. He created winning strategies for Axcom.   (http://math.berkeley.edu/~berlek/). In 1991, He sold the Axcom fund to Renaissance Technologies Inc. and got a very good price for it.

 

 

We can see an interesting pattern here. The scientific research conducted by Berlekamp is basic and theoretical in nature, but his research in the coding theory, cryptography and computer algorithms has given him considerable financial returns. What will happen to his current research of Go ?

 

Simon is a great mathematician, and for many years he has been the Math Department Chair of SUNY-Stony Brook. He studied in UC Berkeley, and got his Ph.D. there. He later returned to Berkeley, and there, together with famous mathematician S.S.Chern, they discovered the Chern-Simon Invariant, which is very important in the area of differential geometry, quantum field theory, and superstring theory.

 

Another famous hedge fund manager, D. E. Shaw, has praised Simon with the following words: “He is a first-rate scholar, with a genuinely scientific approach to trading. There are very few people like him.” D.E. Shaw himself is a very successful hedge fund manager; he created a hedge fund company which bears his name. Starting from a small capital, his current personal assets amount to 2.5 billion dollars, as a result of the successful returns of his funds (according to Forbes’s March 2010 data). David Shaw got his Ph.D in Computer Science from Stanford University, and was an assistant professor at Columbia University before starting his hedge fund.

 

Simons and Shaw are involved in philanthropic activities due to their great success in investment. Simons has made sizable donations to SUNY-Stony Brook and Brookhaven National Lab. Shaw has been interested in sponsoring non-profit biotech research. (As a side note, the original proposal of Amazon was an internal report presented by the then employee Jeff Bezos to D. E. Shaw.)

 

It’s also worth mentioning that for three years Simons conducted research in the field of cryptography for the Institute of Defense Analyses.  Renaissance Technologies Inc. hired many top researchers in the area of natural language processing. Its current CEO was the former natural language expert from IBM. Entropy is an important concept in natural language processing because natural language processing is the study of the amount of information in a paragraph, which is related to the entropy of the paragraph.

 

Simons and Shaw apply science to investment. Simons said, “Fundamental trading gave me ulcers, science, I understood…” Renaissance Technologies hires many researchers with a background in mathematics, physics, engineering or computer science, but they hardly ever hire anyone with a financial background [12, 13]. Because of good returns, this company and other successful quantitative investment firms have created a new job category, Quant. Quantitative investment activities were strong even after the 2008 financial crisis and the Madoff scandal.

 

Hedge funds don’t usually reveal their investment strategies, but it is often reported that the concepts discussed earlier in this article, such as Constant Re-balance Strategies, Kelly Criterion, and high frequency trading were used by some hedge funds in the past.

 

Looking back, entropy, derived from the study of heat, got into the investment field through communication theories. Many people have done well in their investments due to methods that may be related to these scientific concepts, and such efforts have created the quantitative investment categories on Wall Street. Yet, another equation, which is also related to the study of heat, has had an even greater impact on modern financial industry and is highly correlated to the financial crisis that started in 2007. To introduce this equation, let’s review the concept of stock options.

 

 

 

8.  Stock Options and Heat Diffusion Equation

 

In their basic form, stock options give the owner of the option the right to buy or sell a specific stock in the future at a given price. For example, assuming that the current price of stock A is $28, and you bought its three-month call option with the targeted stock price of $40 for $0.02/share from the option market. Three months later you will have the right to buy this stock at $40/share, regardless of its price at that time. If you are very lucky, three months later the stock price will reach $50.  Since you can still buy it at the price of $40, the return on your investment will be: (50-10)/0.02 = 500 times. If after three months the stock price does not go higher than $40, for example, even if it reaches the price of $39.99, then your option will become worthless.

 

Lucky Option Buyer Example: If you are able to buy 1000 contracts of the above call option, each contract corresponding to 100 shares of stocks, then when the stock price reaches $50, your investment will become (50-40)*100,000= $1 million, and you will only have to pay $2000 (ignoring the transaction fee) for these options ($2000 = 100,000*0.02). 

 

Options can be considered as a kind of insurance. The gain in this example is similar to an insurance gain. Someone bought an insurance policy from an insurance company, and later on, the insurance company paid the policy holder a large sum of money.

 

Since you make almost $1 million gain from this investment, it is clear that the option seller lost about the same amount of money. It could naturally be asked why the option seller sells the options at this price. An answer to this question would be that the option seller believes it is almost impossible for the stock price to reach $50  that time frame[14]. If he thinks that there will be a greater probability that the stock price can reach $50, he will demand a higher price for this option. Of course, if his asking price for the option is too high, there will be no buyer. 

 

Clearly, the option price is very important in option trading.  How is the price set? What is the “fair” value of the stock option?  Intuitively, the option price depends on the present stock price , the future targeted stock price ,  time span , interest rate ,  and the “uncertainty ” of this stock price in the future. Among these factors, option price depends on time because the uncertainty and the unpredictability of stock price depend on time. Option price depends on the interest rate because  the initial capital for any future investment can be considered  to be borrowed from a bank, and if the deposit interest rate is very high, people will prefer to put money in a bank for a risk free return rather than  invest it in the stock market.  With the preceding  information, how does the stock price depend on these factors?

 

The answer is given by the famous Black-Scholes-Merton equation. The equation is actually a heat diffusion equation, and  is rooted in the Brownian Motion that was studied by Einstein when he studied the statistical properties of molecular motion. The following is the Black-Scholes-Merton equation for the option we have just discussed [10, 15]:

                                                             (7)

 

where is the price of the stock option,  is a function of time   and the stock price ,  .  is the interest rate,  is a coefficient which describes the “uncertainty of stock price in the future”.  and are assumed to be time-independent constant.  If and depend on time, the equation needs to be modified.

 

With  the given boundary conditions, solving the above problem can give the “fair” prices of the stock options.

 

The Black-Scholes-Merton equation is used to price simple financial derivatives. More complex financial derivatives have their own financial models, which are more complicated. Often these models contain coefficients similar to , to describe the “uncertainty of future price movement”. If the future price variation follows a normal distribution, then  can be used to describe how large the variation is (Figure 2).

 

To price a financial derivative, equations like Black-Scholes-Merton are very important. However, the correctness of the solution and the usefulness of the model depend on coefficients ( or its equivalent) which are hard to estimate [14, 16].  Coefficient  contains the risks of the pricing model.  The difficulty in estimating  is a partial reason for the financial crisis which has almost meltdown Wall Street, and its tremendous impact on the global economy is still felt today.

 

9.  The “uncertainty”  and the financial crisis

 

Stock options are one of the simplest financial derivatives, but even for such simple derivatives, their uncertainty is hard to estimate. The movement of future stock prices depends on many factors, such as the performance of the business, the performance of competitors, the sector that the business belongs to, the overall stock market, the economy, and many other factors. For much more complicated derivatives, such as credit default swaps(CDS), we can imagine that their risks are very difficult to estimate. CDS was considered as one of major financial innovations during the booming years of the US housing market before the housing market collapsed. It is a type of insurance related to trade mortgage backed securities (MBS). MBS is another financial innovation created during the housing boom. MBS was created to make it easy for financial institutions to trade security related to mortgages [17].

 

We will not discuss here the fundamental causes of the financial crisis, for example, that the US economy depends too heavily on the services sector, that its manufacturing sector is losing competitiveness in a world market, or the regulation and policy issues in the financial industry.  We will only focus on some technical causes of this crisis.

 

In simple technical terms, financial institutions such as American Insurance Group (AIG), Bear Sterns, Lehman Brothers and others depend on some models to analyze the uncertainty and risks of derivatives like CDS. How much risk is in CDS trading? Since CDS is a type of insurance related to mortgage trading, clearly, its risks are related to  housing market. Just before the financial crisis, someone asked  one AIG top executive if AIG had considered the risks of the housing market. The executive said: “We have stress tested our models. Even if (overall) housing price drops 10%,  there will be no problems to our models”.  Then the person asked again, “What if (overall) housing prices drop more than 10% ? “ The executive said “This had not happened in history” [19].

 

What happened following the conversation had become history. Similar to the unlucky seller in our luck option buyer example, AIG, as an insurance company, estimated that the risks for CDS trading were  low, so it sold many CDS, which later caused the company great losses. AIG became one example of a “too big to fail” company, and it required the taxpayers to bail it out. Other companies such as Bear Sterns, Lehman Brothers went bankrupt because the risk estimations for their derivatives trading were not correct.

Because of the high leverage and built-in risks in derivatives, Warren Buffet called financial derivatives “Weapons of Massive Destruction” (WMD) [18].

 

The financial crisis further proves the importance of estimating correctly the risks of financial derivatives, (coefficients  in equation (7) or their equivalents in similar models). Being able to estimate risks better and faster than competitors often means great profits and investment returns for hedge funds and institutional investors. This area is often at the forefront of using the latest technologies. In recent years, multi-core CPU, Graphic Processing Unit (GPU) and Cloud Computing have all been used to calculate the risks of financial derivatives. The racing is still on.

 

10. Back to Newton and investment bubbles

 

Before concluding the paper, let’s recall Newton’s quote at the beginning of the paper. Regarding his losses in the South Sea Company bubble, he said: “I can calculate the movement of the stars, but not the madness of men".  He attributed the cause of the investment bubble to the “madness of men” rather than to any fundamental economic or social reasons, such as the demand for oil, “lack of regulations”,  etc.

 

Newton’s observations are interesting.  There have been many investment bubbles in the past. Each time has its own special social and economic reasons. Each time an investment bubble forms, no matter in which geographical region or in which business section, someone will try to point out why this time it is different or why this time it is not a bubble.

 

Often, what really happens is that when a bubble forms, or more specifically, when a bubble approaches its peak, different investment bubbles show similar characteristics, and these characteristics often do not depend on the specific economic or social reasons which trigger the bubble at the first place.

 

Since Newton era is too far away from us, let’s take a look at three recent investment bubbles represented by their market index. The first one is the NASDAQ index, which approached its peak (about 5000 points) in the spring of 2000.  The second is the Shanghai stock A share index, which approached its peak (about 6000 points) at the end of 2007, and the third one is the Japan Nikkei 225 index, which approached its peak (about 38000) at the end of 1989. Each of these bubbles happened for its own reasons. At the end of the 1980’s, Japan’s economy was very strong.  In 2000, the US was riding the Internet wave and new “innovations” appeared every day.  In 2007, China’s economy was very strong because of its powerful manufacturing sectors and exporting engines.  Figure (3) plots three indexes when they are around their peaks. The X-axis shows the number of trading days away from the day when the indexes reached their individual peaks. The day that an  index reached its peak is set to 0; the Y-axis is rescaled index points representing the daily closing points of each index divided by its own peak value, therefore the maximum value of those rescaled index points is 1.  

 

In Figure (3), the three investment bubbles show somewhat similar trends when they approach their peaks, particularly, the last 200 trading days before each index reached its peak. Why do investment bubbles have such characteristics? Is it because when investment bubbles approach their peak, they are governed by some kind of “many body effect”, which is independent of the underline economic/social reasons?  Or in Newton’s words, is  the bubble caused by “the madness of men”?

 

A well studied “many body effect” in physics, the critical phenomena, show some similar characteristics to investment bubbles. Critical phenomena take place when some physical systems approach to their critical points during second order phase transition. For example, the ferromagnetic phase transition is one example of critical phenomena.  Figure (4A) and (4B) show a two dimensional model of a lattice system before and after ferromagnetic phase transition, where each arrow represents the direction of the magnetic pole (North pole or South Pole) of a molecular (or an atom) at a lattice. Figure (4A) shows the state of the system before the ferromagnetic phase transition,  the magnetic poles randomly point up or down and the system does not show net macro magnetic properties. The model is referred to as 2D Ising model in physics [20]. The model further assumes that only short range interactions exists in the system, which means that each molecule (or  atom) can only impact its nearest neighbor and often such interactions can be assumed to be  weak. Figure (4B) shows the state of system after ferromagnetic phase transition,  the magnetic poles uniformly   point to one direction and the system show net macro magnetic properties.

 

Critical phenomena in physics studies when and how a many body system can change from the state in Figure (4A) to the state in Figure (4B). When a system approaches  the state in Figure (4B), referred to as a critical point, a  set of  well chosen parameters, commonly referred to as critical exponents,  are used to describe the system properties under such transition.

 

It turns out that systems show the following  properties: (1) long-range correlations although the underlying interactions can be short-range; (2) the divergence of critical exponents at critical points; (3) different systems can show similar behaviors when they approach  critical points, and such behaviors are independent of their underlying interactions (the concept of “universality” in physics). Because of long-range correlations, a small local fluctuation can cause big global ripples, and may trigger a phase transition.  The analogies of these properties in investment bubbles are:  people make similar investment decisions during an investment bubble; the price rise often accelerates when a bubble approaches its peak; and different invest bubbles show similar characteristics. Starting from those similarities, a natural question to ask  is, “Can we use similar models of critical phenomena in physics to study investment bubbles?”

 

An intuitive approach is to use the above Ising model to model people’s investment decisions.  Assume many people sit in a brokerage house, let an arrow  represent each person’s investment decision. For example, an up arrow  is for ‘buy’ and a down arrow is for ‘sell’, Figure (4A) would show a state in which people’s investment decisions are random and not correlated; Figure (4B) would show a state in which people make almost same investment decision, all wanting to buy or to sell. When this happens, we can expect an investment bubble (or an investment panic). This is be one of the simplest models of the investment bubble and it is completely independent of underlying economic reasons which trigger a bubble. We can study if such a simple model can really be used to study some financial bubbles. If so, , we can study when and how an investment bubble really happens, and what will trigger and prevent the formation of a bubble.

 

Investment bubbles may happen on a global scale or on local ones. They can happen in specific sectors, such as real estate, technology and commodities. Because an investment bubble often causes misallocation of social resources, the burst of a major investment often induces negative economic impact. It would be useful to be able to predict the formation of a bubble when a small froth appears.  For institutional and individual investors, being able to make such predictions also means superior investment returns. Yet making such predictions is not easy, because very often froth disappears and does not form large bubbles. Investment panics, which are the opposite of investment bubbles, often show behavior similar to the behavior of investment bubbles.

 

Another interesting question regarding investment bubbles is whether the market will still be efficient when a bubble is approaching its peak. James Simon said: "Efficient market theory is correct in that there are no gross inefficiencies, but we look at anomalies that may be small in size and brief in time.”  Will a bubble amplify such anomalies [21] ?

 

Conclusion

 

The article introduces connections between physics, communication technologies and investments. It presents how entropy enters quantitative investments and the difficulty in estimating the risks of financial derivatives which caused the recent financial crisis. It also points out that investment bubbles show some similarities to “many body effect” of critical phenomena in physics. Figure(5) shows people, concepts, and connections discussed in this article.

 

Given the deep impact the financial crisis and investment bubbles can have on world economy and people’s life, the concepts presented in this paper are worth further investigation.  These concepts will also be useful to people in the fields of quantitative investments and financial modeling.

 

Author: Bin Lee,  Research area: signal processing for extremely noisy.  Ph.D EE Stanford University.  e-mail: bin.lee @ gmail . com

 

 

Reference:

  1. Spence, Anecdotes, 1820, p. 368
  2. CLAUDE ELWOOD SHANNON, Collected Papers, Edited by N.J.A Sloane and Aaron D. Wyner, IEEE press, ISBN 0-7803-0434-9
  3. Solomon W. Golomb, et al. Claude Elwood Shannon (1916-2001), Notices of AMS, Jan 2002
  4. Thomas Starr, Massimo Sorbara, John M. Cioffi, and Peter J. Silverman, DSL Advances, Prentice Hall PTR (2003)
  5. B.Lee, et al. Gigabit DSL, IEEE Transactions on Communications, Sep 2007
  6. Thomas M. Cover, Joy A. Thomas,  Elements of Information Theory, 2nd Edition, Wiley, 2006
  7. William Poundstone, Fortune's Formula, Hill and Wang, 2005
  8. J. L. Kelly, A New Interpretation of Information Rate. Bell System Technical Journal, Vol 35, pp 917-926
  9. E. Thorp, The Kelly Criterion In Blackjack, Sports Betting, and the Stock Market, The 10th International Conference on Gambling and Risk Taking, June 1997
  10.  David G. Luenberger, Investment, Oxford University Press, 1998
  11. P. A. Samuelson, Why We Should Not Make Mean Log of Wealth Big Through Years to Act are Long , Journal of Banking and Finance 3, (1979) 305-307 
  12. Hal Lux, The Secret World of Jim Simons, Institutional Investor Magazine, Nov 1, 2000
  13. Jim Simons on "From Quant to Riches", May 2007
  14. Taleb, Nassim Nicholas, The Black Swan: The Impact of the Highly Improbable, Random House (2007)
  15. John Hull, Options, Futures, and Other Derivative Securities, 6th ed., Prentice Hall
  16. Roger Lowenstein, When Genius Failed: The Rise and Fall of Long-Term Capital Management, Random House Trade, 2001
  17. J.B. Foster, F. Magdoff, The Great Financial Crisis : Causes and Consequences, Monthly Review Press, 2009
  18. Berkshire Hathaway Letters, 2002
  19. Quote of an AIG senior manager
  20. Huang, Kerson , Statistical mechanics (2nd edition), Wiley, ISBN 0471815187, 1987
  21. Burton G. Malkiel, Patricia A. Taylor, From Wall Street to the Great Wall, W.W.Norton & Company 2008

 

 

 

                    

 

 

                    

 

 

                  Sigma_Redraw

Figure 2：Normal distribution and

 

..\Support_Materials\Stock_Compare_Chart_EnglishVersion_SingleColor.jpg

Figure 3：Investment Bubble Comparing, Nasdaq Index, Japan Nikkei 225 Index,

Shanghai A Share index near their historical peaks

 

 

 

 

 

                   

              

 

 

 

 

 

 

 

 

 

 

 

 

 


Figure 4A,  Directions of Arrows, Random                          Figure 4B, Direction of Arrows, Uniform 

Before a phase transition or before an invest bubble             After a phase transition or in an invest bubble

 

 

Figure 5：From Newton to Financial Crisis

Concepts, people and connections discussed in this paper
 
 

 

 

 

 


Download PDF version of this document: Financial Signal Processing, Bubble Detection, Reflection Detection.pdf

Chinese version of Document are here.

Download Chinese PDF Version: Financial Signal Processing, Bubble Detection, Reflection Detection.pdf