Why App Engine
Google App Engine enables you to build web applications on the same scalable systems that power Google applications. App Engine applications are easy to build, easy to maintain, and easy to scale as your traffic and data storage needs grow. With App Engine, there are no servers to maintain: You just upload your application, and it’s ready to serve to your users.
App Engine is a complete development stack that uses familiar technologies to build and host web applications. With App Engine you write your application code, test it on your local machine and upload it to Google with a simple click of a button or command line script. Once your application is uploaded to Google we host and scale your application for you. You no longer need to worry about system administration, bringing up new instances of your application, sharding your database or buying machines. We take care of all the maintenance so you can focus on features for your users.
You can create an account and publish an application that people can use right away at no charge from Google, and with no obligation. When you need to use more resources, you can enable billing and allocate your budget according to your needs. Find detailed pricing for usage that has exceeded the free quota on our Billing page.
For the first time your applications can take advantage of the same scalable technologies that Google applications are built on, things like BigTable and GFS. Automatic scaling is built in with App Engine, all you have to do is write your application code and we’ll do the rest. No matter how many users you have or how much data your application stores, App Engine can scale to meet your needs.
The Reliability, Performance, and Security of Google’s Infrastructure
Google has a reputation for highly reliable, high performance infrastructure. With App Engine you can take advantage of the 10 years of knowledge Google has in running massively scalable, performance driven systems. The same security, privacy and data protection policies we have for Google’s applications applies to all App Engine applications. We take security very seriously and have measures in place to protect your code and application data.
Google App Engine made a splash when it launched in the spring of 2008. It was different from most other cloud systems back in the day because it was neither IaaS (Infrastructure-as-a-Service, e.g., Amazon EC2) nor SaaS (Software-as-a-Service, e.g., Salesforce). It was something in-between and ushered in the era of PaaS (Platform-as-a-Service). Instead of a fixed application (SaaS) or raw hardware (IaaS), App Engine managed your infrastructure for users. Furthermore, it provided a development platform… users get to create apps, not used the one provided by the cloud vendor, and it leveraged the infrastructure as a hosting platform.
The development-to-release cycle is minimized because high-level services that developers would normally have to build are already available via an App Engine API. A development server is provided to let users test their code (with certain limitations) before running in production. And finally, deployment is simplified as Google handles that all. Outside of setting up an account and billing structure, there is no machine setup or administration as Google takes care of all logistics there too. Even as your app is running with fluctuating network traffic, the App Engine system auto-scales up to allocate more instances of your app as needed, then similarly releases resources when no longer needed.
A web developer can now use Google’s infrastructure, finely tuned for speed and massive scaling, instead of trying to build it themselves. In the past, developers would create an app, generally need a machine or web hosting service that could host a LAMP stack, administer each of the “L”, “A”, “M”, and “P” components, and somehow made the app globally accessible. Moreover, developers were also generally responsible for the load-balancing, monitoring, and reporting of their systems, and to reiterate one of the most difficult and expensive things to build yourself: scaling. All of these are taken care of by App Engine.
By now, you have a good idea as to why Google developed App Engine. To put it simply, to remove the burden of being a system administrator from the developer. Using a LAMP stack involves choosing a distribution of Linux, choosing of the kernel version, etc., configuring PHP and an Apache web server. There is also the need to run and manage a database server (MySQL or otherwise) and other aspects of a running system (monitoring, load-balancing, reporting). The list continues with managing user authentication, applying software patches and performing upgrades, each of which may break your app, bringing even more headaches for developers/sysadmins.
Other than the actual application, everything else is nearly orthogonal to the solution that developers are trying to create for their customers. App Engine attempts to handle these complexities to let you focus on your app(s). An app running on App Engine should be easy to build, manage, and scale.
PaaS and what isn’t App Engine?
Some users confuse Google App Engine with Amazon’s EC2 service. The problem is that this is an apples to oranges comparison. Both operate at different cloud service levels, and each have their strengths and minuses. With App Engine, you only need to worry about your application and let Google take care of hosting and running it for you. With EC2, you’re responsible for the app, but also its database server, web server, operating system, monitoring, load-balancing, upgrades, etc. This is the reason why typically, the costs for IaaS services run lower than that of PaaS services because with PaaS, you’re “outsourcing” more work/responsibility. Cost estimates usually clouded by not considering the administration overhead when managing the infrastructure yourself. A better “apples-to-apples” comparison would be EC2 to the Google Compute Engine IaaS service.
PaaS systems also differ from that of the SaaS layer above, as SaaS applications are fixed and must be taken directly from the respective cloud vendor. Unless you work with or at the vendor, you cannot change the SaaS application you use. It’s quite the opposite with PaaS systems because you (as a PaaS user) are the developer, building and maintaining the app, so the source code is your responsibility. One interesting perspective is that you can use a PaaS service to build and run SaaS apps!
Security is gravely important. Developers (typically) would not be interested in letting other applications/users get any kind of access to their application code or data. To ensure this, all App Engine applications run in a restricted environment known as a sandbox.
Because of the sandbox, applications can’t execute certain actions. These include: open a local file for writing, open a socket connection, and make operating system calls. (There used to be more restrictions, but over time, the team has tried to bump up quotas and remove as many restrictions as they can. These don’t make the airwaves as much as bad news does.)
Any network developer would say, “Without being able to support two-way socket connections, you can’t create useful applications!” The same may be true of the other restrictions. However, if you think about it carefully, why would you want use these lower-level OS features? “To make useful apps with!” You want to use outbound sockets to talk to other processes, and perhaps you may want inbound sockets to listen for service requests.
The good news is that the App Engine team knows what you want, so that’s why the team has created a set of higher-level APIs/services for developers to use. Want your app to send and receive e-mail or instant messages? That’s what the e-mail and XMPP APIs are for! Want to reach out to other web applications? Use the URLfetch service! Need Memcache? Google has a global Memcache API. Need a database? Google provides both its traditional NoSQL scalable datastore or access to the relational MySQL-compatible Google Cloud SQL service.
The list of all the services that are available to users changes quite often as new APIs are created. At the time of this writing, the following services/APIs are available to developers:
- App Identity
- Cloud SQL
- Cloud Storage
- Federated Login (OpenID authentication)
- (Full-Text) Search
- NDB (new database)
- OAuth (authorization)
- Prospective Search
- Task Queue (Pull and Push)
- Users (authentication)
You can read more about most of these APIs in the official APIs docs pages. (Docs for the others are available but not on this page.) Also, The Google App Engine team is constantly adding new features, so keep your eyes on the Google App Engine blog for announcements on new and updated services and APIs.
One of the benefits in choosing to host your apps on PaaS systems is being freed from administration. However, this means giving up a few things… no longer do you have full access to your logs or be able to implement custom monitoring of your app (or your system). This is further impacted by the sandbox runtime environment mentioned above.
To make up for some of this lack of access to application and system information, Google has provided various tools for you to gain a better insight into your app, including its performance, traffic, error rate, etc.
The first tool is an administration console. (App Engine provides two “admin consoles” actually.) A fully-featured version is for your application running in production while the other one is a lightweight version for the development server.
. The team has added so many new features that the current incarnation of the dashboard includes far more features than are illustrated here. However, the general structure and information displayed is relatively the same.
Another tool is a general system status page, While it is not an indication of how any one particular app is doing, it does show what is going on with the system as a whole.
The final tool is Appstats. It is in the same class as a profiler but custom-made to help you determine inefficient ways your code may be interacting with App Engine services (rather than traditional profiling of code coverage, memory usage, function call metrics, program behavior, etc.). Its specific use is best described in App Engine team’s introductory blogpost:
“Appstats is a library for App Engine that enables you to profile your App Engine app’s performance, and see exactly what API calls your app is making for a given request, and how long they take. With the ability to get a quick overview of the performance of an entire page request, and to drill down to the details of an individual RPC call, it’s now easy to detect and eliminate redundancies and inefficiencies in the way your App Engine app works.”
Applications (web & non-web)
While many applications running on Google App Engine are web-based apps, they are certainly not limited to those. App Engine is also a popular backend system for mobile apps. When developing such apps, it’s much safer to store data in a distributed manner and not solely on devices which could get lost, stolen, or destroyed. Putting data in the cloud improves the user experience because recovery is simplified and users have more access to their data.
For example, the cloud is a great place for mobile app user info such as high scores, contacts, levels/badges, etc. If users lose their phone, they would only need to get a new phone and reinstall the application. All their data can be streamed from the cloud after that. Not only is recovery simplified, but it makes possible scenarios like users being able to pull up their leveling or high score info from the home computer upstairs in the evenings while their mobile phones charge downstairs. Again, cloud can be a tool to provide a better user experience!
When developing a backend for mobile applications, the same decision needs to be made on whether a company should host it themselves or take it to the cloud. Do you spend time and resources building out infrastructure, or is it better to leave it to companies that do this for a living and focus on the application instead?
Mobile phones only need to be able to make outbound HTTP requests to instantly connect to your App Engine backend. You can control the application-level protocol, authentication/authorization, and payload contents, so it’s not any more complex than providing a similar backend for a traditional web app.
Migration to/from App Engine
The section title alone cannot convey all the aspects of this category when considering cloud vendors. It includes migration of applications to/from your target platform (in this case App Engine), ETL and migration of data, bulk upload and download, and vendor lock-in.
Porting your applications to App Engine is made simpler by providing familiar development environments, namely Java, Python, and now Go. Java is the de facto standard in enterprise software development, and developers who have experience building Java servlets will find App Engine to be quite similar. In fact, Java App Engine’s APIs adhere as closely with existing JSR (Java Specification Request) standards as possible.
In addition to the servlet standard (JSR-154), App Engine supports the JDO and JPA database interfaces (or you can choose to use Objectify or the low-level interface directly). If you’re not comfortable with NoSQL databases yet, you can use Google Cloud SQL, the MySQL-compatible relational cloud database. The App Engine URLfetch service works like the Java SE java.net.URL class, the App Engine Mail API should work just like the javax.mail (JSR-919) class, the App Engine Memcache API is nearly identical to using the javax.cache (JSR-107) class, etc. You can even use JSP for your web templates.
On the Python side, while Google ships a lightweight web framework (webapp/webapp2) for you to use, you aren’t limited to it. You can also use: Django, web2py, Tipfy, Bottle, and Pyramid, to name some of the more well-known frameworks that work with Python. Furthermore, if you have a Django app and use the third-party Django-nonrel package (along with djangoappengine), you can move pure Django apps onto App Engine or off App Engine to a traditional hosting service supporting Django with no changes to your application code outside of configuration. For users choosing Cloud SQL instead of App Engine’s traditional NoSQL datastore, you can use Django directly as there is an adapter specially written for Cloud SQL.
Next are the SDKs. For all supported runtimes, they are open source. This allow users to become familiar with the App Engine client-side libraries and possibly build their own APIs. In addition, if users desire to control the backend, they can use the SDK and the client APIs to come up with corresponding backend services. Not comfortable letting Google host and run your app(s)? This gives you an alternative. In fact, there are already two well-known App Engine backend projects: AppScale and TyphoonAE. Both claim they are 100% API-compatible, meaning that any App Engine app that Google can run, they should be able to as well.
Next, you have control over your data. When using the traditional datastore, Google provides a datastore bulkloader. This tool lets you easily upload or download all of your data. You can find out more about the bulkloader in the official docs. Other features in App Engine related to your data include backup/restore, copying, or deleting your data. Find out more about those also in the official docs. Similarly, if using Google Cloud SQL, you can easily import or export your data using Google Cloud Storage as an intermediary. You can read more about that at the Cloud SQL docs on import/export.
Finally, with all the advantages of a PaaS system like Google App Engine, some developers may wonder about “vendor lock-in,” a situation describing how it may be difficult or impossible for companies to move their apps and/or data to similar or alternative systems. While Google would love you to stay as an App Engine customer for a long time, Google recognize that having choices makes for a healthy and competitive ecosystem.
If Google is a cloud vendor and App Engine is its product, does vendor lock-in apply? Well, yes and no. Think of it this way: you use Google’s infrastructure to avoid having to build it yourself. Arguably this is one of the most difficult and time-consuming things to do. So you can’t get something for nothing. The price you pay is that you need to integrate your app so that it connects to all of App Engine’s components.
However, while Google recommend that you code against Google App Engine’s APIs, there are workarounds. Also, think back to why you want to use App Engine… for its robustness and scalability. Google created our APIs so you could take advantage of Google’s infrastructure and not as a deliberate attempt to force you into using Google’s APIs.
By allowing alternative ways of accessing your data, using familiar development environments, following open standards, and distributing open source SDK libraries, Google fights vendor lock-in on your behalf. It may not be easy to migrate, but Google has implemented features to make it easier to migrate your app or upload/download all of your data. Google tries hard to ensure that you can move your apps or data onto or off of App Engine. But it doesn’t stop there… the team is continually innovating, listening to user input, and improving and simplifying App Engine services to further provide a development environment of choice for building your web (and non-web) apps. Finally, the best practices you’ll learn in creating great App Engine apps can be used for application development in general, regardless of the execution platform.
Other Important Features
The final two remarks here pertain mostly to enterprises. Google App Engine is proud to be compliant with global security standards. Google are certified SAS 70 compliant as well as compliant for its replacements SSAE 16 and ISAE 3402.
Enterprises who are Google Apps customers will find App Engine to be an extremely useful tool. When you purchase Google Apps, Google provide a default set of applications that help you run your business. In addition to the apps from Google, there are also many more built by third-party software firms that you may find compelling in the (Google Apps Marketplace]
If none of the applications above meet your needs, you can use Google App Engine to build your own custom applications and roll them into your Google Apps domain and manage them from the same control panel as if you bought it from Google directly or from vendors in the Apps Marketplace.