Recent Posts

Pages: 1 ... 3 4 [5] 6 7 ... 10
Google Cloud Platform: How Preemptible Instances Can Bolster Your Cloud Cost-Optimization Strategy

Back in August last year, Google Cloud Platform announced price reductions of up to 33% on its Preemptible Virtual Machines (VMs), which could deliver potential savings of up to 80% on the cost of its regular on-demand instances.

The move was just the latest in a long series of price cuts designed to attract more enterprise customers to the platform and challenge the dominance of Amazon and Microsoft in the cloud infrastructure marketplace.

But what exactly are Preemptible VMs? And what applications are suitable for deployment to the service? In this post, we run through the main features of Preemptible VMs and compare them with Amazon’s rival discount offering Spot Instances.

What Are Preemptible VMs?

Launched in May 2015, Preemptible VMs are a disposable class of instances through which Google offers excess compute capacity at a much lower price compared with its standard machines.

They’re just like their regular VM counterparts, except that they automatically shut down after 24 hours. What’s more, at any time during that period, they may be shut down at short notice whenever the vendor needs to reclaim spare capacity to meet compute demands elsewhere.

Google takes a number of factors into consideration in its preemption process. For example, it avoids stopping too many VMs from a single customer and gives preference to instances that have been running longer. In other words, instances are more at risk of preemption when they first start running. However, apart from any separate licensing costs, you’re not charged if Google stops your instance within the first 10 minutes.

Charges for Preemptible VMs are typically around 20% of the full on-demand price. Based on average usage, which takes into account partial sustained-use discounts, they generally work out at about 25% of the running cost of the equivalent standard machine. The service carries no guarantee of availability and no SLA.

What Applications Are a Good Fit?

Preemptible VMs are a great way to reduce the cost of processing large-scale workloads that aren’t time sensitive and require massive amounts of compute resources. They’re well suited to security testing, short-term scaling, media encoding, financial or scientific analysis and insurance risk management, where you can spin up hundreds or thousands of machines and quickly complete a job in a short space of time.

They’re a particularly useful option as part of the VM mix for massively parallel processing applications, such as Hadoop and Spark, as they can complement Google’s conventional VMs to enhance performance. However, they’re not suitable for mission-critical services such as operational databases and Internet applications.

How Should Your Workloads Handle Preemption?

First of all, if you haven’t done so already, you should build fault tolerance into your application. You can also mitigate against the impact of preemption by combining regular instances with Preemptible VMs in your clusters, thereby ensuring you maintain a baseline level of compute availability.

Alternatively, you can create a shutdown script that responds to preemption alerts by automatically launching regular instances to cover your shortfall in compute capacity. But, if costs are paramount and you’re prepared to wait, your script should simply clean up and save your job so it can pick up where it left off.

In either case, you should test your application’s response to a preemption event. You can do this by stopping the VM and checking that your script correctly completes your shutdown procedure.

However, if you want to avoid preemption in the first place, you should look to run your instances at off-peak times, such as nights and weekends, when the risk of disruption is at a minimum.

How Do They Compare with Spot Instances?

Preemptible VMs are very similar to Amazon’s Spot Instances. Nevertheless, there are several key differences between them.

Preemptible VMs are charged at fixed rates, which are individually set according to the type and size of instance. Availability depends on the vendor’s level of spare capacity.

By contrast, allocation of Spot Instances is based on a bidding process, where your machine will run as long as your bid price is above the current Spot price on the Spot Market. Provided you don’t cancel your request, the machine will restart whenever the Spot price falls below your bid price. And while your machine is running the actual price you pay is the Spot price.

Some enterprise customers will prefer the more predictable costs of Preemptible VMs. On the other hand, some will like the flexibility of Spot Instances—through the fact they don’t automatically terminate after 24 hours and that you can bid at a higher price to reduce the risk of interruption.

Preemptible VM costs are rounded up to the nearest minute. However, if you terminate your instance within the first 10 minutes, usage is rounded up to a full 10 minutes. By contrast, the costs for Spot Instances are rounded up to the nearest hour. But, if your Spot Instance is interrupted, you don’t pay for your last partial hour of usage.

Finally, Google only gives you 30 seconds’ notice of preemption, while Amazon gives you 2 minutes’ notice of interruption. And Preemptible VMs are available to any Compute Engine instance type, whereas Spot Instances aren’t available to the burstable T2 family of instances.

As yet, there is no Microsoft equivalent of either service.

A Highly Capable Offering

Google is a relative newcomer to the cloud computing arena and still playing catch-up with Amazon and Microsoft in terms of available features and market share. Nevertheless, it has already established itself as a highly capable offering and an ideal cloud platform for specific use cases.

However, in the company’s drive to attract new enterprise customers, pricing will continue to play an important role in its marketing strategy. So, if your applications are a good fit for the platform, you can expect further cost-cutting campaigns and even better value for money from your cloud infrastructure expenditure. And all the more so if you optimize your costs, adopt a flexible approach to your workloads and take advantage of cost-saving options such as Preemptible VMs.

Published by Jonathan Maresky
Published on Virtual Strategy Magazine

Announcements / Qube! 6.9-1 Released
« Last post by jburk on February 16, 2017, 07:55:50 PM »
This is a maintenance release of the Qube! Core/Supervisor/Worker/ArtistView/WranglerView products.

This is a recommended release for all customers running Qube v6.9-0.

Notable changes and fixes are:
  • Numerous memory leaks in the supervisor have been plugged, fixes runaway memory consumption of single supervisor processes.
  • Faster re-loading of the qbwrk.conf central worker configuration file, also optimizes supervisor startup.
  • qbping results always reflect the current state of license file, artificially low values for installed licenses were throwing off the "metered license" calculations.

Please see for complete release notes.
Google circulates a new whitepaper that appears designed to reassure Google Cloud Platform customers that their data is protected by multiple layers of physical and cyber-security.

Data security and trust have long been major concerns for organizations considering cloud-computing options.  In an apparent bid to allay these fears among its customers at least, Google has released a new whitepaper enumerating the complex multi-layered strategy the company uses to protect enterprise data in the cloud.
The paper shows that Google has deployed security controls in six progressive layers starting with physical and hardware security at the bottom and operational security controls at the top of the stack.

A lot of the technology in Google's data centers is home built and incorporates what the company claims are multiple physical security controls.

Access to Google's data centers is tightly restricted and only a "very small fraction" of Google employees ever have access to the facilities housing the systems that power the company's range of cloud computing services. Security measures for controlling facility access include biometric identification, laser-based intrusion detection systems, vehicle barriers, metal detection and webcams.

Full story by By Jaikumar Vijayan
Published here

Rendering in the Cloud / Survey: Google Cloud Most Popular Choice for SMBs
« Last post by Render Guru on January 12, 2017, 07:30:04 PM »
From Kris Blackmon | The VAR Guy
Published here:

A new survey from Clutch may explain why enterprises gravitate toward Microsoft Azure while smaller organizations choose Google.

A recent Clutch survey of 247 organizations showed that while powerhouse cloud providers Amazon Web Services (AWS) and Microsoft Azure tend to be the top choices of enterprise customers, small and midsize businesses (SMBs) gravitate toward Google Cloud Platform (GCP).

The data was collected from businesses from one to 10,000+ employees, and respondents were evenly distributed among users of each service, with about a third from each. Across all three platforms, "better selection of tools/features" ranked as the top reason customers chose their primary provider, with brand familiarity and security tying for second.

The Rise of the Public Cloud and the Future of the Channel
Public Cloud Is Imminent: Three Vital Migration Tips for ISVs

Despite being the oldest provider and having the lion's share of the market, AWS ranks lowest on brand familiarity. According to the survey, it ranks at 15 percent. Azure was the most recognized brand at 24 percent, and GCP sat right at at 20 percent.

The survey found that 37 percent of Azure users identify as enterprises, compared to only 25 percent who identify as an SMB and 22 percent who call themselves a startup or sole proprietorship. In contrast, 41 percent of GCP users fall into the SMB category.

Nick Martin, Principal Applications Development Consultant at Cardinal Solutions, says enterprise loyalty to Azure makes sense. “Windows Server and other Microsoft technologies are prevalent in the enterprise world. Azure provides the consistency required by developers and IT staff to tightly integrate with the tools that Microsoft-leaning organizations are familiar with.”

The report theorizes that GCP's pricing may be more palatable to SMBs, which when combined with its brand familiarity, may explain its popularity in that space. However, it's notable that GCP's analytics tool, Cloud Datalab, is the provider's most popular service, suggesting that smaller businesses may use it as their sole analytics service.

Clutch draws some high-level conclusions from its data that may help partners that are migrating customer data from on-prem to a public cloud:

*If you are an enterprise, require Windows integration, or seek a strong PaaS (platform-as-a-service) provider, consider Microsoft Azure.

*If you want heavy emphasis on analytics or are an SMB with a limited budget, look into Google Cloud Platform.

*If a service’s longevity, IaaS (infrastructure-as-a-service) offerings, and wide selection of tools are important to you, Amazon Web Services may be your best option.
Full Story by Tara Seals US/North America News Reporter, Infosecurity Magazine

Published here:

Google is broadening its continuum of encryption options available on Google Cloud Platform (GCP), with the addition of the Cloud Key Management Service (KMS).

Now in beta, Cloud KMS offers a cloud-based root of trust that customers in regulated industries, such as financial services and healthcare, can monitor and audit. As an alternative to custom-built or ad-hoc key management systems, which are difficult to scale and maintain, Cloud KMS is aimed at making it easy to keep keys safe.

“With the launch of Cloud KMS, Google has addressed the full continuum of encryption and key management use cases for GCP customers,” said Garrett Bekker, principal security analyst at 451 Research. “Cloud KMS fills a gap by providing customers with the ability to manage their encryption keys in a multi-tenant cloud service, without the need to maintain an on-premise key management system or HSM.”

With Cloud KMS, users can manage symmetric encryption keys in a cloud-hosted solution, whether they’re used to protect data stored in GCP or another environment. Users also can create, use, rotate and destroy keys via our Cloud KMS API, including as part of a secret management or envelope encryption solution. It’s directly integrated with Cloud Identity Access Management and Cloud Audit Logging for greater control as well.

“Forward thinking cloud companies must lead by example and follow best practices,” said Maya Kaczorowski, Google product manager, in a blog. “For example, Ravelin, a fraud detection provider, encrypts small secrets, such as configurations and authentication credentials, needed as part of customer transactions, and uses separate keys to ensure that each customer's data is cryptographically isolated. Ravelin also encrypts secrets used for internal systems and automated processes.”

Leonard Austin, CTO at Ravelin, added, “Google is transparent about how it does its encryption by default, and Cloud KMS makes it easy to implement best practices. Features like automatic key rotation let us rotate our keys frequently with zero overhead and stay in line with our internal compliance demands. Cloud KMS’s low latency allows us to use it for frequently performed operations. This allows us to expand the scope of the data we choose to encrypt from sensitive data, to operational data that does not need to be indexed.”

At launch, Cloud KMS uses the Advanced Encryption Standard (AES), in Galois/Counter Mode (GCM), the same encryption library used internally at Google to encrypt data in Google Cloud Storage. This AES GCM is implemented in the BoringSSL library that Google maintains, and continually checks for weaknesses using several tools, including tools similar to the recently open-sourced cryptographic test tool Project Wycheproof.

By default, Cloud Storage manages server-side encryption keys, but if users prefer to manage their cloud-based keys themselves,  they can select Cloud KMS. For managing keys on-premise, they can select Customer Supplied Encryption Keys for Google Cloud Storage and for Google Compute Engine.

“While we’re on the topic of data protection and data privacy, it might be useful to point out how we think about GCP customer data,” added Kaczorowski. “Google will not access or use GCP customer data, except as necessary to provide them the GCP services.”

Announcements / Qube! WranglerView 6.9-0c patch release is available
« Last post by jburk on December 15, 2016, 01:52:40 AM »
A 6.9-0c patch release is now available for the Qube 6.9-x WranglerView.  This is a recommended patch release for all customers running any version of Qube v6.9, and can be installed directly on top of any 6.9 version of Qube.

For those using the Qube Installer utility, the 6.9-0 manifest file has been updated.  If you are using version 2.1-0 or later of the QubeInstaller installer utility, it will auto-detect if a manifest has a newer version and offer to download it.

You may also download the patched version directly at:

Full release notes are available on our documentation site:

Code: [Select]
This is a WV-only release to roll-up bug fixes that were impacting several customers
New features
< None >

==== CL 17323 ====
@FIX: default user can't remove their own jobs: userHasQubePermission: Unknown Qube user permission: "remove"
@FIX: non-admin users should only be permitted to remove jobs that are either complete, failed, or killed
==== CL 17323 ====
@FIX: right-click on WV job list is slow on Windows

Changes in behavior
==== CL 17339 ====
@CHANGE: WV will exit at startup when running against a supervisor from an older major/minor version
We have released a patched version of Qube! core, supervisor, and worker packages, labeled 6.8-4a, that contain various fixes.

The Qube! Installer should automatically pick up this new version when the 6.8-4 manifest is selected.

Listed below is the RELEASE notes for 6.8-4a, for your reference.


Code: [Select]
@RELEASE: 6.8-4a

This is a cumulative patch release of the qube-core, supervisor, and worker
packages, for all platforms, including several key fixes.

==== CL 17208 ====
@CHANGE: Popluate the subjob (instance) objects with more data (like status), and not just the IDs, when subjob info is requested via "qbhostinfo" (qb.hostinfo(subjobs=True) for python API)

Previously, only jobid, subid, and host info (name, address, macaddress)
were filled. Now, things like "status", "timestart", "allocations",
etc. are properly filled in.

ZD: 16541

==== CL 17206 ====
@FIX: When "migrate_on_frame_retry" job flag is set, prevent backend from doing further processing (especially another requestwork()) after a work failed

This was causing race-conditions that will get agenda items to be stuck in
"retrying" state, while there are no instances processing them.

Now the reportwork() API routine is modified so that if it's invoked to
report that a work "failed", and the "migrate_on_frame_retry" is set on the
job, it will stop processing (does a long sleep), and let the worker/proxy
do the process clean up.

ZD: 16553

==== CL 17186 ====
@FIX: "VirtualBox Host-Only Ethernet Adapter" now when daemons (supe, worker) try to pick a primary mac address

ZD: 16561

==== CL 17182 ====
@CHANGE: all classes that inherit from QbObject print as a regular dictionary, no longer have a __repr__ which prints the job data as a single flat string
@NEW: add qb.validatejob() function to python API, help find malformed jobs that crash the user interfaces

==== CL 17141 ====
@FIX: Any job submitted from within a running job picks up the pgrp of the submitting job

By design, if the submission environment has QBGRPID and QBJOBID set, the
API's submission routine will set the job's pgrp and pid, respectively to
the values specified in the environment variables.

One couldn't override this "inheritance" behavior even by explicitly
specifying "pgrp" or "pid" in the job being submitted, for instance with
the "-pgrp" command-line option of qbsub.

Fixed, so that setting "pgrp" to 0 on submission means that the job should
generate its own pgrp instead of inheriting it from the environment.

ZD: 16545

==== CL 17101 ====
@NEW: add "-dying" and "-registering" options to qbjobs.
@CHANGE: also add dying and registering jobs to the "-active" filter.

ZD: 16469

==== CL 16804 ====
@TWEAK: added code to print what operation was requested, when printing out "permission granted to user..."

Announcements / PipelineFX public repository and FTP server scheduled maintenance 10/21
« Last post by jburk on October 20, 2016, 10:35:54 PM »
The public repository which hosts the manifest files and installer packages (MSI, RPM, DMG) used by the Qube installer will be offline for maintenance for approximately 2 hours beginning 18:00 PDT (UTC -7) October 21st.  The following services will be unavailable or unreachable during this time:

The Qube Installer will not be able to reach the PipelineFX public repository during this time, but will still be able to perform installations, upgrades, etc from any local repository you have configured.  If you anticipate doing any installations or upgrades during this time, you may use the Installers "Download Only" option before the scheduled maintenance begins to populate your local repository.

Announcements / Patched versions of Qube! 6.9-0a and 6.9-0b released
« Last post by pipelinescott on October 13, 2016, 02:12:41 AM »
On Monday, we released a patched version of Qube! 6.9-0, labeled as 6.9-0a, which contained a fix for thumbnails and image preview in ArtistView.

Today, on Wednesday, we subsequently released another patched version of Qube!, labeled as 6.9-0b, which contains a fix for qbping which could have affected users as early as Qube! installation.

If any further issues are seen, please do contact support and let us know right away.

We strongly encourage the downloading and use of the latest patched version. The manifests and release docs have been updated accordingly.

Rendering in the Cloud / Google introduces Google Cloud and G Suite
« Last post by Render Guru on October 07, 2016, 04:40:35 PM »
Google unveiled some important announcements, designed to help businesses build, grow, and compete with cloud technology. You can read the complete announcement on our blog post, but here are some highlights:

  • Google for Work is now called Google Cloud.
The new name captures our mission to bring businesses everywhere the best cloud technology and services on the planet.

  • Google Apps has a new name, too
Pages: 1 ... 3 4 [5] 6 7 ... 10