Browsing: / Home
Shortlink

Project with 18K lines of code in a class (Challenges) – Part II

By Prabhu Sunderaraman on July 21, 2019 in , , , ,

In the previous post, we discussed about the nature of the application highlighting lack of some key items. Let’s discuss the deployment infrastructure of the application.
The application is deployed in a jboss domain cluster. There are three nodes in the cluster with a load balancer in the front. Each JBoss server has an Apache Web server that serve’s as a Reverse Proxy. There are a couple of other standalone jboss servers that are used for tasks to run jobs, perform some tasks like document conversions etc., A couple of identity servers used for authentication and authorization implemented using Shibboleth.
The application is file IO intensive. So there’s a SAN based storage volume created just for storing files and the volume is mounted on each server. And there’s ElasticSearch instance and MySql instance in the background.

Screen Shot 2019-07-21 at 7.52.41 PM

So far so good!!! Before we jump into the rationale behind using Jboss, domain clusters etc, let’s talk about the challenges.

Running out of Memory

The first and foremost problem is servers running out of memory. Each server is 16 core processor 32GB RAM, bare-metal machine and not a VM. The JVM memory utilization hits 90% every 2 days and stops responding. So there’s a manual restart required. The servers are hosted in a localized infrastructure provider. For a specific set of customers we had the infrastructure in AWS as well.

Cluster crash

When one of the instances is down, the other two instances keep handling the requests. But when you try to bring up the instance that went down, the other two instances also crash. So everytime we’re forced to restart the whole cluster, which eats up sweet 15 minutes as there is no automated way to do this as I mentioned in the earlier post.

Running out of Space in SAN volume

The SAN storage area eats up so much data to the tune of approximately 1 TB per month. So maintaining it, backing it up, cleaning it up is a huge mess. Often it goes out of space and the application becomes unresponsive. Increasing the size or reclaiming is a tedious task.

The Load balancer issue

Ideal job of load balancers is to balance out the load on the servers. It scans all the incoming requests and directs them to the appropriate servers that are capable of handling the requests at any point of time.

This application had a specific problem based on a particular file upload functionality. The front end implemented a file upload functionality using jQuery uploader plugin. When the size of the file is huge, the implementation automatically splits into chunks and uploads the file. So roughly a 10 MB file gets uploaded as 7-8 chunks to the server. In the server, after the chunks are uploaded, a background job merges all the chunks and pushes them to the storage area.

So this is how it works in a non-load-balancer environment.

Screen Shot 2019-07-21 at 7.38.14 PM

Let’s wear our thinking hat on for sometime before we delve into the challenges of this particular functionality in a load-balanced environment in our next post.

Mixx Delicious Digg Facebook Twitter
Shortlink

Project with 18K lines of code in a class – Part I

By Prabhu Sunderaraman on July 17, 2019 in , , , ,

I have been consulting for a company say X, the last few months. X is one of those companies that has a product used by several clients, with approximately 100K transactions per day, but with code that has become unmaintainable, very hard to make changes. I got a call from them, asking for my services in improving the performance, migrating to the cloud, moving their application to microservices architecture and building a team.
Having been there, done that, I always get into these consulting assignments with not a pinch of salt but bagful of salt.

blog#1

My first day at their premises turned out to be one of the memorable days in my career. After customary handshakes and product explanations, the team emphasised they wanted to improve the performance as they are forced to restart their servers every 3 days. The application has been built by a number of people in the past, passed on to so many others and now being maintained and worked upon by few hapless developers. It’s a familiar story in software development. Isn’t it?

After sometime the curious programmer in me took over and I told them that I wanted access to their codebase. That’s when the unexpected happened. I got a mail to a link to Google Drive, with access to the entire code. It was 500MB+ Java code. Experience has always taught me to be a silent observer during the start of these assignments. So I understood in a no time about the nature of the application.

  • It didn’t have a proper version control mechanism. SVN was being used as recycle bin. So no Version Control strategy
  • No MAVEN/GRADLE. Yes you read it right!!! Every jar file was manually downloaded and copied in the project. I didn’t want to get into the Whys? of this
  • ZERO test cases. I don’t get surprised at all these days when I come across applications with no test cases. I have long before come to accept that it requires a certain amount of skill and maturity from developers and managers to understand the importance of having test cases.
  • No automated builds/deployments. Every two weeks application was deployed manually in all the servers in QA and production environment
  • Monolith application, with 400MB of WAR file built and deployed
  • Spring application with framework concepts of Spring being used/abused extensively.

Now getting inside the guts of the application, here’s what I found.

  • There are approximately 700 JSPs.
  • Each JSP has 500 lines of code on an average. It’s a spaghetti of jQuery, JSP scriptlets, CSS and HTML code very hard to read
  • Tens of thousands of classes
  • The best is here. Each class has on an average 5000 lines of code. The longest class had 18000 lines of code. And there were easily a hundreds of such classes. During my course of stay, when I set up Git and tried to open one such class, GitHub politely refused to open saying “It’s too long to open”
  • Code written in Java 8, but not even a single class has any feature related to Java 8 used.
  • Oh, duplication of code, thy name is X

Now, please go back and re-read the first paragraph of this post.

.
.
.
.
.
Throughout these discussions, I kept hearing words like scrum, agile, stand-up meeting that only Dilbert came to my mind.

blog#2

Inspite of these shortcomings, the deployment infrastructure was the saviour which held this whole application together. But it was a double-edged sword. Let’s talk about this in the next post.

Mixx Delicious Digg Facebook Twitter
Shortlink

ReactJS video series – State with React – (Part 2)

By Prabhu Sunderaraman on February 8, 2019 in ,

This is the eighth part of the video series on React JS.

We continue our discussions on the basics of state in React JS. State represents the data model of a component. It’s bound to the UI. Technically, it’s just JSON data. Changing the state data using setState() method invokes the render() method. We have a button in this example, clicking which modifies the state variable, resulting in UI update

The example used in the video is given below 

<html>
    <head>
        <script crossorigin src="https://unpkg.com/react@16/umd/react.production.min.js"></script>
        <script crossorigin src="https://unpkg.com/react-dom@16/umd/react-dom.production.min.js"></script>
        <script src="https://unpkg.com/babel-standalone@6/babel.min.js"></script>
    </head>
            
    <body>
        <div id="root"></div>
        <script type="text/jsx">
          class Sample extends React.Component {
              constructor(props) {
                  super(props);
                  this.state = {
                      currentTime: new Date()
                  };
              }

              getCurrentTime() {
                  let currentTime = new Date();
                  //setState method is used to update any item in the state
                  this.setState({
                      currentTime
                  })
              }

              render() {
                  return (<div>
                      <h1>Time now: {this.state.currentTime.toString()}</h1>
                      <button onClick={this.getCurrentTime.bind(this)}>Get Current Time</button>
                  </div>);
              }
          }
          ReactDOM.render(<Sample/>, document.getElementById("root"));  
        </script>
    </body>        
</html>

You can find the video here.

Read and watch the other parts here.

1. Hello ReactJS without JSX
2. Hello ReactJS with JSX
3. Let’s play with JSX – Part I
4. Let’s play with JSX – Part II
5. Creating Components – Part I
6. Creating Components – Part II
7. State with React – Part I
Mixx Delicious Digg Facebook Twitter
Shortlink

ReactJS video series – State with React – (Part 1)

By Prabhu Sunderaraman on February 7, 2019 in ,

This is the seventh part of the video series on React JS.

We discuss the basics of state in React JS. State represents the data model of a component. It’s bound to the UI. Technically, it’s just JSON data.

The example used in the video is given below 

<html>
    <head>
        <script crossorigin src="https://unpkg.com/react@16/umd/react.production.min.js"></script>
        <script crossorigin src="https://unpkg.com/react-dom@16/umd/react-dom.production.min.js"></script>
        <script src="https://unpkg.com/babel-standalone@6/babel.min.js"></script>
    </head>
            
    <body>
        <div id="root"></div>
        <script type="text/jsx">
          class Sample extends React.Component {
              constructor(props) {
                  super(props);
                  this.state = {
                      currentTime: new Date()
                  };
              }

              render() {
                  return (<div>
                      <h1>Time now: {this.state.currentTime.toString()}</h1>
                  </div>);
              }
          }
          ReactDOM.render(<Sample/>, document.getElementById("root"));  
        </script>
    </body>        
</html>

You can find the video here.

Read and watch the other parts here.

1. Hello ReactJS without JSX
2. Hello ReactJS with JSX
3. Let’s play with JSX – Part I
4. Let’s play with JSX – Part II
5. Creating Components – Part I
6. Creating Components – Part II
Mixx Delicious Digg Facebook Twitter
Shortlink

ReactJS video series – Creating Components (Accessing UI Components and Event Handling) – (Part 2)

By Prabhu Sunderaraman on January 23, 2019 in ,

This is the sixth part of the video series on React JS.

We discuss accessing UI components and event handling in React JS. Let’s continue with the login form we built in the previous post.

The example used in the video is given below 

<html>

<head>
    <script crossorigin src="https://unpkg.com/react@16/umd/react.production.min.js"></script>
    <script crossorigin src="https://unpkg.com/react-dom@16/umd/react-dom.production.min.js"></script>
    <script src="https://unpkg.com/babel-standalone@6/babel.min.js"></script>
</head>

<body>
    <!--Let's build a login form-->
    <div id="root"></div>
    <script type="text/jsx">
        class LoginForm extends React.Component {

            constructor(props) {
                super(props);
                
                //Works with the latest version of React
                this.username = React.createRef();
                this.password = React.createRef();
            }

            loginButtonClicked() {
                //Access username and password textboxes
                //NOT RECOMMENDED
                // let userName = document.getElementById("username").value;
                // let password = document.getElementById("password").value;

                let userName = this.username.current.value;
                let password = this.password.current.value;
                console.log(userName, password);
            }

            render() {
                return (<div>
                    <input type="text" placeholder="User name" ref={this.username}/>
                    <br/>
                    <input type="password" placeholder="Password" ref={this.password}/>
                    <br/>
                    <button onClick={this.loginButtonClicked.bind(this)}>Login</button>    
                </div>);
            }
        }
        ReactDOM.render(<LoginForm/>, document.getElementById("root"));
    </script>
</body>

</html>

You can find the video here.

Read and watch the other parts here.

1. Hello ReactJS without JSX
2. Hello ReactJS with JSX
3. Let’s play with JSX – Part I
4. Let’s play with JSX – Part II
5. Creating Components – Part I
Mixx Delicious Digg Facebook Twitter