Amazon Web Services Blog

AWS Job: Architect & Designer Position in Turkey

2009-01-16T10:21:00-08:00

Late last year an entrepreneur from Turkey visited me at Amazon HQ in Seattle. We talked about his plans to use AWS as part of his new social video portal startup. I won't spill any beans before he's ready to talk about it himself, but I will say that he has a really good concept, strong backers, and infectious enthusiasm for the online world.

He's now ready to hire a software architect and designer in order to bring his vision to life. I've posted the job below; you can send your resume to apply@web.tv if you are interested, qualified, and located in the right part of the world.

Software Architect & Designer

We are a reputable Internet technology, software services and e-commerce company based in Istanbul and Bursa, Turkey. We are looking for a talented Software Architect who will be working in Istanbul for a certain period of time, for our new global scale "social video portal" project. Below are the qualifications required and job description for the position to be held.

Qualifications:

Extensive knowledge of web technologies.
Experienced in web based application design and development.
Solid bacground in object oriented design and development.
Preferrably experienced in live broadcasting over the internet, video streaming, video sharing and social networking web site development and design.
Knowledge and experience of design and development of multi-tier, distributed, massively multi-user systems.
Experienced in Cloud Computing applications (preferably with AWS).
Very good command of PHP or Python.
Experinced in relational database design.
Familarity with Erlang, and knowledge or experience of Java, C/C++, Ajax, Adobe Flex, mySQL is a plus.
Self motivated, enthusiastic, team player.

Job Description:

Will be mainly responsible for designing the overall system for a multi-tier, massively multi-user live video multi-casting, videosharing web site which will also have features of a social network.
Will be involved in Design and Development phases of software development cycle. Will contribute to the Analysis phase.
Will lead the Software Development Team for the period of the contract and report to the Project Coordinator.

-- Jeff;

Mainframes in the Cloud?

2009-01-15T07:11:16-08:00

Almost! Micro Focus just deployed a managed mainframe emulation environment that runs on top of Amazon EC2. The beauty of this environment is that you can execute CICS or IMS code in the cloud, essentially unmodified. Micro Focus’ innovation has three significant benefits: (a) your existing code investment is protected, (b) costs are much, much lower than an in-house mainframe, and (c) the application execution environment is managed, complete with the same sort of service level agreements that are expected in the mainframe world.

Mark Haynie, the Micro Focus CTO of Application Modernization, says that in his opinion “Cloud computing is about services, not languages”. Since Micro Focus Enterprise Cloud Services allows COBOL CICS and IMS applications run in the cloud as easily as they run in on-premises datacenter, the consumer of those services will not know or care what language they were written in.

Darryl Taft at eWeek also wrote an article about this, which you can read here.

The announcement is about more than bringing mainframe applications to the Amazon Cloud. In my opinion, it is also another example of an entire ISV ecosystem that is cloud-enabled. Because Micro Focus’ own development community is supported by Enterprise Cloud Services, there is a new business opportunity for many developers that blends tradition with innovation. It’s easy to forget that mainframes are at the core of many enterprises; and for that reason I believe that interest will be significant. You can learn more at cloudservices.microfocus.com.

SimpleDB Developer's Brown Bag

2009-01-12T11:24:00-08:00

You are invited to join the Amazon SimpleDB team on Tuesday, January 20, 2009 at 9am PST for the first session of our new Developers’ Brown Bag. During these once monthly webinar sessions, developers will hear from the technical experts behind SimpleDB, and have the opportunity to engage in live Q&A.

Interested developers may register by emailing simpledb-developer-brownbag@amazon.com. Please include name & AWS Account ID. Here's where to find your AWS Account ID (that's not my real ID so don't try anything funny):

In addition, developers are encouraged to pre-submit any questions they may have, to allow for a more thorough response during the live webinar. For those struggling with the development of a new application, sample code and a description of the intended application may also be submitted for review and discussion.

The team is looking forward to speaking with developers on the 20th.

-- Jeff;

CloudFront Management Tool Roundup

2009-01-09T17:51:58-08:00

Amazon CloudFront was designed to make it really easy to distribute content to users at high speed with low latency. Here are some new tools which provide a nice end-user interface to CloudFront.

The newest Freeware release of the CloudBerry Explorer now includes CloudFront support. You can create and manage distributions, assign CNAMES, and even automate the entire process using the Windows PowerShell. CloudBerry Explorer also includes some powerful support for batched changes to S3 object Access Control Lists. There are a couple of helpful videos here.

StreamInCloud is a free FLV (Flash Video) encoder. You simply create an S3 bucket and give StreamInCloud permission to read and write it. It then monitors the bucket for new videos, encodes them into the FLV format, and places the encoded version in the bucket. Of course, if the bucket is part of a CloudFront distribution, the encoded content is then available worldwide at high speed with low latency.

StreamInCloud encodes the videos at 512kbps and leaves the size as-is. This service is free; an advanced version with additional features and options will be available later at an additional charge.

Cyberduck is a Mac OS X client for Amazon S3 and CloudFront, with added support for FTP, SFTP, WebDav, and other online storage facilities. The product has a very long feature list, is "scriptable via AppleScript, and, like CloudBerry Explorer, is Freeware.

Full source code is available as well.

As I noted earlier this week, Ylastic allows you to manage your CloudFront distributions from your iPhone. There's now support for the Google Android Phone as well. Watch the screencasts to learn more.

Affirma Consulting has developed the Manager For Amazon CloudFront in C#. The project is hosted on CodePlex and full source code is available. It supports direct streaming of data into S3 and uses multiple threads to manage simultaneous uploads, downloads, and live statistics.

On the surface, CloudBuddy looks like a free S3 bucket explorer tool with full support for CloudFront. However, there's quite a bit more beneath the surface. It is actually a platform with a highly refined architecture. All CloudBuddy operations are exposed as APIs.

The distribution includes a Microsoft Office plug-in to help you to manage your documents, workbooks, emails, presentations, and projects in the cloud. Source code is available.

Bucket Explorer also has a number of unique and very handy features including the ability to copy objects from one S3 account to another along with timed backups to S3. It is available for Windows, Mac, and Linux.

Enjoy, and let us know how you have put CloudFront to use.

-- Jeff;

Manage Amazon EC2 With New Web-Based AWS Management Console

2009-01-08T19:04:40-08:00

Today we’re announcing the availability of the Web-based AWS Management Console, which in this first release provides management of your Amazon EC2 environment via a point-and-click interface. A number of management tools already exist: for example a popular Firefox extension known as Elasticfox; however as you read more of this post I believe you’ll agree that the new console is compelling--especially when it’s time to log in as a new AWS developer.

For starters, it’s easier than ever to gain access to your Amazon EC2 environment. The console provides access via your Amazon username and password. No more certificates or public/secret keys to manage! If you’re like me, I never seem to have my own computer at hand when I need to check the status of the Amazon EC2 farm, or for that matter when I need to launch a new instance. It’s a lot easier to log in with a username and password than to use those same credentials to retrieve my keys, configure Firefox (if it’s even on the borrowed computer) and then log in.

Then there’s the new point-and-click AJAX user interface for managing Amazon EC2 resources. No more page refreshes every time something updates; and a timer refreshes management console components, such as the status of running instances, every few seconds.

The AWS community creates an amazing selection of innovative Amazon Machine Images, or AMIs. In fact, the count is now a staggering 1200 AMIs and growing! That’s quite a menu to choose from—especially if you are a first-time user. The new Launch Instance Wizard walks you through starting your first instance; offering a short list of Linux and Windows server choices. Choose one of these AMIs, and then the wizard even suggests which ports to open in the firewall. It’s smart enough to suggest that you open SSH (port 22) for Linux images, and RDP (port 3389) for Windows instances. The wizard even suggests settings that restrict Amazon EC2 access to “your computer only”.

And as I hinted in the opening paragraph, this is just the first in a set of Console interfaces that will provide a UI layer on top of AWS infrastructure services. We’ll be adding additional Amazon Web Services in the future.

The console feature list is extensive, and provides intuitive management of all these things:

AMI Management: browse and search AMIs, launch instances from AMIs, deregister and register AMIs
Instance Management: launch, reboot, terminate, get console output, RDP/SSH help, etc.
Security Group Management: create and delete security groups, add and remove permissions, configure firewall settings, open and close ports
Elastic IP Management: create and release IP Addresses, associate IPs to instances
Elastic Block Store: create, delete, attach and detach volumes. Take snapshots and manage snapshots.
Key Pair management: create and delete public/private key pairs.

If you’d like to take a six minute tour of the console, I created a video (Flash format).

Finally, do you have a feature suggestion, or some other type of feedback? Feedback links are at the bottom of each page, and we welcome your input.

Mike

Cloud Computing and Biomedical Research Roundtable in San Diego

2009-01-08T05:47:59-08:00

Cirrhus9 (mentioned yesterday) and Pfizer are co-sponsors of a roundtable discussion on the topic of cloud computing and biomedical research. Amazon CTO Werner Vogels will be in attendance at this unique event, where they'll discuss the emerging demands of biomedical research and how they can be met using cloud computing.

The roundtable will be help at 2 PM on January 14th at the Pfizer building in San Diego.

There are just 7 seats left so you'd better go ahead and register now.

-- Jeff;

AWS Links - Wednesday, January 7, 2009

2009-01-07T11:09:12-08:00

As usual, I've got plenty to blog about. Here's a glimpse at some of the interesting things that have recently landed in my inbox.

On Thursday, January 8th, Information Week and Amazon will present a Webcast titled How To Plug Into The Cloud. Attendees will learn how Eli Lilly has used Amazon's EC2 servers and S3 storage to support its pharmaceuticals research. Presenters will include Dave Powers from Eli Lilly and Adam Selipsky from Amazon Web Services. Attendance is free but you do need to preregister.

The folks at rPath will be conducting a series of tech events on alternate Tuesdays starting on January 13th. Attendees at the first event will learn how to build a virtual appliance using rBuilder Online and launch it on Amazon EC2. Once again, attendance is free but preregistation is a must!

Content Workspace has built a document scanning and indexing service around Amazon S3 and the Amazon Mechanical Turk. Access to the Mechanical Turk workforce gives them the ability to offer document and form classification, document indexing, data entry, translation, detection of duplicates, and more.

I met Chris Richardson a year or two ago at a conference in Philadelphia. Since then we've stayed in touch and have bumped in to each other at conferences on the east and west coast. His Cloud Tools project makes it easy to deploy, test, and manage Java EE applications on EC2. Now, Chris has created the Cloud Foundry in order to make this process even easier. You can simply drop your application's ".war" files into a container managed by Cloud Factory and running on EC2. You can choose between a number of runtime topologies and can manage your entire cluster with one button. Learn more by watching the Cloud Foundry screencast.

Mike from Cirrhus9 wrote to tell about his company and their cloud computing service offerings. They focus on the Life Sciences industry, with 3 of the 4 founders coming from the pharmaceuticals business. They got together as a "Cloud Integrator" to help IT managers navigate and leverage cloud computing. Their offerings include an evaluation methodology to evaluate the feasibility of moving individual applications to the cloud. They also understand the FDA Qualification and Validation process and can work with companies that would like to move health care applications into the cloud.

They've also created CloudBuntu, a lightweight managed desktop running on EC2 and accessible through a web browser.

By way of the Ylastic blog comes news that their very full-featured AWS management application can now be run on an Apple iPhone. Using Ylastic you can architect, manage, and monitor your AWS cloud computing environment. Watch this application in action on the Ylastic screencast.

I now have proof that cloud computing can be hereditary! My son Stephen is studying Applied Math and Economics at the University of Washington. He's also working on a really interesting research project in the Economics department. The project makes use of the Mechanical Turk in a very unique way. The paper is still being written and I'll be sure to link to it when it finally arrives. Over the holiday break Stephen needed to parse some large XML files. The parsing was taking forever on his laptop, so he launched an Extra Large EC2 instance and did his work there instead. In his post, he explains what he did and also provides a photo-illustrated tour of how he set up EC2 and EBS.

Finally (I am out of time, not material) Researchers in the Physics Department at the University of Washington has prepared a report on the topic of Scientific Computing in the Cloud. They have investigated the feasibility performing scientific computing in the cloud (using EC2) as an alternative to traditional computational tools. They note that "For research groups, cloud computing provides convenient access to reliable, high performance clusters and storage, without the need to purchase and maintain sophisticated hardware. For developers, virtualization allows scientific codes to be optimized and pre-installed on machine images, facilitating control over the computational environment."

And that's it for today. Send me information about what you are doing with AWS and I'll do my best to fit you in.

--Jeff;

Bits For Sale - The New Amazon S3 Requester Pays Model

2009-01-02T14:34:15-08:00

We rolled out a powerful new feature for Amazon S3 in the final hours of 2008.

This new feature, dubbed Requester Pays, works at the level of an S3 bucket. If the bucket's owner flags it as Requester Pays, then all data transfer and request costs are paid by the party accessing the data.

The Requester Pays model can be used in two ways.

First, by simply marking a bucket as Requester Pays, data owners can provide access to large data sets without incurring charges for data transfer or requests. For example, they could make available a 1 GB dataset at a cost of just 15 cents per month (18 cents if stored in the European instance of S3). Requesters use signed and specially flagged requests to identify themselves to AWS, paying for S3 GET requests and data transfer at the usual rates — 17 cents per GB for data transfer (even less at high volumes) and 1 cent for every 10,000 GET requests. The newest version of the S3 Developer Guide contains the information needed to make use of S3 in this way.

Second, the Requester Pays feature can be used in conjunction with Amazon DevPay. Content owners charge a markup for access to the data. The price can include a monthly fee, a markup on the data transfer costs, and a markup on the cost of each GET. The newest version of the DevPay Developer Guide has all of the information needed to set this up, including some helpful diagrams. Organizations with large amounts of valuable data can now use DevPay to expose and monetize the data, with payment by the month or by access (or some combination). For example, I could create a database of all dog kennels in the United States, and make it available for $20 per month, with no charge for access. My AWS account would not be charged for the data transfer and request charges, only for the data storage.

I firmly believe that business model innovation is as important as technical innovation. This new feature gives you the ability to create the new, innovative, and very efficient business models that you will need to have in order to succeed in 2009!

-- Jeff;

Please Vote: AWS and AWS-Powered Applications Are Crunchies Finalists

2008-12-29T12:14:58-08:00

AWS is a finalist in two categories of the Crunchies awards, Best Enterprise and Best Overall.

In addition to that, AWS-powered applications from Animoto and SlideRocket are candidates for Best Design; eBuddy and Fotonauts are up for Best International; DropBox for Best New Startup of 2008; and Twitter (competing with AWS) for Best Overall. GoodGuide is a finalist in the Most Likely To Make The World A Better Place category, as is Akoha. Meebo is a candidate for Best Application.

If you could take the time to vote that would be great! Here are some direct links to make it really easy:

Best Enterprise - AWS

Best Overall - AWS

Best Overall - Twitter

Best Design - Animoto

Best Design - SlideRocket

Best International - eBuddy

Best International - Fotonauts

Best New Startup of 2008 - DropBox

Most Likely To Make The World A Better Place - GoodGuide

Most Likely To Make The World A Better Place - Akoha

Best Application - Meebo

-- Jeff;

PS - If I missed any AWS-powered applications, drop me a note and I'll update this post!

Solve for Efficiency With Amazon Mechanical Turk

2008-12-29T12:10:35-08:00

This is a quick post about another great use of Amazon Mechanical Turk. Over on Friend Feed Jean-Claude Bradley posted a blog post called “Mechanical Turk Does Solubility on Google Spreadsheet”, which talks about using Mechanical Turk to process solubility data for the Open Notebook Science Challenge.

What I believe is revolutionary here is that rather than rely on government funding to help process results, scientists are able to use Crowdsourcing to process results in real time—and at a very fine unit of granularity. When you combine those characteristics with the Open Notebook Science Challenge, the result is unprecedented transparency.

The blog post referenced above inspired another, titled “ Generalizability coefficient for Mechanical Turk annotations”, that describes using Amazon Mechanical Turk and Crowdsourcing to generate annotations of a thesis. Wow, eliminating (or more likely reducing) one of the biggest pains associated with a thesis. Doctoral candidates everywhere should take note.

It’s about more than Grad Student pain relief though: by Crowdsourcing mechanical tasks such as annotation, students are free to focus on data collection and analysis. That makes the entire process more efficient.

These innovative uses of Amazon Mechanical Turk combined with the new Public Data Sets, are an exciting time in both Cloud Computing and in science.

Mike