tag:blogger.com,1999:blog-9238405.post2558118378473105113..comments2024-03-18T02:04:50.380-07:00Comments on Agile Testing: Experiences with Amazon Elastic MapReduceGrig Gheorghiuhttp://www.blogger.com/profile/17863511617654196370noreply@blogger.comBlogger5125tag:blogger.com,1999:blog-9238405.post-43593272031428011122012-08-29T12:52:35.397-07:002012-08-29T12:52:35.397-07:00Great post, thanks for sharing your experience!
C...Great post, thanks for sharing your experience!<br /><br />Can you please give us more details on how you insert date into a MySQL instance? Thank you.Patricenoreply@blogger.comtag:blogger.com,1999:blog-9238405.post-17774788403316338052012-04-20T09:35:04.576-07:002012-04-20T09:35:04.576-07:00Hi Jie Li -- we still run our EMR cluster 24x7 bec...Hi Jie Li -- we still run our EMR cluster 24x7 because our data analyst needs to run ad-hoc reports during the day too. It still makes economic sense for us, cheaper than investing in our own Hadoop cluster.Grig Gheorghiuhttps://www.blogger.com/profile/17863511617654196370noreply@blogger.comtag:blogger.com,1999:blog-9238405.post-80829588750915311132012-04-19T20:40:09.668-07:002012-04-19T20:40:09.668-07:00Interesting! What is your current workflow now? If...Interesting! What is your current workflow now? If you only need to process 4 hours a day, then the "elastic" workflow might save money. But seems the reserved instances might be cheaper for the long run, say, 3 years?Jie Lihttps://www.blogger.com/profile/10356578452947361776noreply@blogger.comtag:blogger.com,1999:blog-9238405.post-9620762811766075502011-11-07T11:35:42.426-08:002011-11-07T11:35:42.426-08:00Every night we process around 100 GB of data. It t...Every night we process around 100 GB of data. It takes around 4 hours, including the download from S3 to HDFS, then the Hive queries (which are fairly complex and include joins).Grig Gheorghiuhttps://www.blogger.com/profile/17863511617654196370noreply@blogger.comtag:blogger.com,1999:blog-9238405.post-71726369350554031952011-11-05T23:53:22.775-07:002011-11-05T23:53:22.775-07:00How much data are typically processing and how lon...How much data are typically processing and how long does it take to process it? We have some similar use cases as well, but I *think* our needs might be more real time oriented.<br /><br />I had forgotten all about EMR since it was released, so thank you for the simple example.Anonymousnoreply@blogger.com