Crm Server Side - Unable to Read Data From Transport
Ever since Server-side tagging was publicly announced at SUPERWEEK 2020, Google and the trusted tester customs take been hard at work, building something that just might modify the landscape of digital analytics for good.
Google Tag Manager has now released Server-side tagging into public beta. In this lengthy article, nosotros'll take a look at what Server-side tagging is, how it should (and should not) exist used, and what its implications are on the broader digital analytics community.
In short, Server-side tagging means running a Google Tag Manager container in a server-side surround (at the time of writing, the only bachelor environment is the Google Cloud Platform, though I'thousand certain more options will become available in good time).
Many of the benefits and concerns are tackled in their respective chapters. Even so, I desire to emphasize that Server-side tagging has the potential to overturn the current dynamic of data collection and governance for an organization. Y'all ain and have total control over the server-side surround. You have admission to tools and methods to thoroughly vet and validate the traffic betwixt network sources and your advertising and analytics endpoints.
You can run a fully functional digital analytics and marketing setup without loading any third-political party code in the user'southward browser or device. With advisable monitoring in identify, y'all can say goodbye to PII and credential leaks, cross-site tracking traps, and bloated 3rd-political party JavaScript encumbering the client.
Server-side tagging utilizes many of the concepts familiar to Google Tag Manager users:
-
There are tags which burn down on triggers and pull in data from variables.
-
New container versions can be previewed and published.
-
Users can create their own custom templates.
Yet, there are new, fundamentally unlike features that introduce something of a paradigm shift to the type of dynamic tagging that Google Tag Manager promotes.
-
The container itself is a new Server blazon; different from the web, app, and AMP containers that precede it.
-
Instead of trigger events, processes are initialized past incoming HTTP requests.
-
These requests are digested by a new type of GTM entity: a Client.
-
The Client parses the requests, generates an event data object, and feeds this into a virtual container, where tags can utilise this result object to map and send data to their endpoints.
This commodity will not exist an exhaustive guide. I will walk you lot through the primary concepts of Server-side tagging and there should be petty you'll be left wanting, but to complement this article, I do recommend you consult Google's ain, official documentation.
Table of Contents
10 The Simmer Newsletter Subscribe to the Simmer newsletter to get the latest news and content from Simo Ahava into your email inbox!
How to follow this guide
While I hope everyone would devour every concluding word of this article, I'm aware that non all sections are relevant to all readers.
If you're looking for an overview of Server-side tagging, possibly for getting buy-in within your organization, I recommend reading these chapters:
- What is Server-side tagging
- Central benefits - Reduced client load
- Key benefits - Content Security Policy
- Key benefits - Total control and buying of the data collected past the container
- Key concerns - All chapters
- Technical outline - Toll
- Technical outline - Custom domain
- Summary
If yous're a developer or working in It, I'd recommend focusing on these chapters:
- What is Server-side tagging
- Key benefits - Reduced customer load
- Central benefits - Keep keys and secrets safety
- Central benefits - More control over what endpoints collect
- Key benefits - Content Security Policy
- Key concerns - All chapters
- Technical outline - Server container
- Technical outline - Custom domain
- Technical outline - Clients and tags
- Technical outline - Custom templates
- Technical outline - Resources
- Summary
Everything else is still important, simply I'll forgive y'all if you gloss over them initially, only to render to them hungrily once you're hooked into all the awesomeness that Server-side tagging brings in its wake.
I recommend yous watch the following 2 videos regardless.
The first one is a general introduction to Server-side tagging, focusing on deployment and getting started with your first Client and tag.
The second is a deep-dive into building your own Client template. It's a fleck more than specialized and can thus be skipped if you're non interested in customizing the container.
Video: Introduction to Server-side tagging
If the video doesn't work, you can lookout it here.
Video: Create a Client template in a Server container
Annotation! The video beneath has 1 of import omission. When creating the Client template, brand sure to update the "Sends HTTP Requests" permission to include "Allow Google Domains". Otherwise the proxying of analytics.js doesn't piece of work.
If the video doesn't work, you can watch it here.
What is Server-side tagging?
With Server-side tagging, Google Tag Manager has introduced a new Server container type, which resides in a Google Cloud environs.
In a nutshell, the purpose of this setup is to create an endpoint in a server environment that you ain. It will act every bit a sort of a proxy between the hits sent from browsers and devices and the actual endpoints to which the hits are nerveless. See the adjacent chapter for more details on what this blazon of proxy can practise.
The container itself operates as an HTTP API endpoint, to which any browser, device, or other sources that support the HTTP protocol can send requests.
Ideally, this endpoint would exist mapped with a custom subdomain in the same domain hierarchy as the website sending the requests. That way the requests are considered to happen in first-party context, which has a significant touch on how cookies can be read and written, for example.
Inside the Server container, workers known as Clients are configured to listen for these incoming HTTP requests, which they and then parse into a unified issue format. The Clients then run a virtual container with the event data object, where tags, triggers, and variables react to the event push like to how they would with "regular" Google Tag Managing director.
Tags accept the data in these event data objects and compile them into HTTP requests to their respective endpoints. Finally, the Customer sends an HTTP response back to the source of the initial request.
All of the above happens inside the confines of the Server-side tagging environment. The just way the browser or app sending the data tin exist fabricated aware of what'due south going on is if the Client adds information into the HTTP response, which is fully configurable.
Central benefits
Here are some of the key benefits of using Server-side tagging.
Reduced customer load
By running the logic of building and dispatching hits to the vendor endpoint in your server-side environs, yous accept a golden opportunity to reduce the amount of (especially third-party) JavaScript run in the user'south browser.
Considering you tin can configure the Server container to map any incoming HTTP request into the format required by the vendor, you can theoretically reduce your entire third-political party pixel and JavaScript load to a single event stream directed into your Server container.
This stream tin can then exist intercepted by a Client which gain to map the stream into the event model expected by the vendor tags, also running in the Server container.
This is the ultimate benefit of a Server-side tagging setup. Even if yous don't want to reduce everything to a single stream, you tin can build your ain custom template in the web container, which builds the HTTP request to the Server container without having to load whatever 3rd-party JavaScript at all (apart from the GTM library itself).
Keep keys and secrets condom
By transporting data processing logic away from the device, where it would exist visible for anyone with debugging skills, you will also be able to run secured and credential-based transactions without having to worry about exposing sensitive information to the device.
For example, a plague on Google Analytics has been Measurement Protocol spam, where malicious parties crawl potential tracking IDs and and so proceed to spam them with automated HTTP requests that masquerade as "regular" hits from the site. Alternatively, these hackers ship spam hits to random tracking IDs, knowing that if they send enough hits, some of them will finish up in real Universal Analytics accounts.
This type of spam is notoriously difficult to identify and prevent considering information technology'due south built to resemble bodily hits that are sent from the website itself.
At present that you have the server endpoint handy, you can add a new Custom Dimension within the Server container, which is so sent to Google Analytics. In Google Analytics, you lot tin can and then create a filter for this Custom Dimension, allowing only traffic that matches it.
consequence['x-ga-mp1-cd11'] = 'my_secret_key';
By adding this "secret key" in the Server container, there's no mode that a random Measurement Protocol spammer can know it's there. Similarly, it won't help if the spammer crawls your site, looking at the requests sent to Google Analytics, because there are no such requests! There are only requests to your own server-side endpoint, and it would be odd if Measurement Protocol spammers would utilise those to fuel their spam algorithms.
Naturally, this isn't express to just what you can do with Universal Analytics. Any tertiary-political party servers that place your admission with an API fundamental or credential token can now be proxied through your Server container and so that these keys are non exposed in the device!
More control over what endpoints collect
Considering your proxy now resides between the user's device and the endpoint, y'all are in full command over what is shipped to the vendor.
Unless the Client specifically overrides things like the IP address and User-Agent in the approachable HTTP request from the Server container (this is what the born Universal Analytics client does by default), the IP and User-Amanuensis cord volition exist that of your Server container rather than the user. So this is a great mode to concretely anonymize this aspect of the HTTP protocol that'due south proven to be problematic in terms of end-user privacy.
Server-side tagging introduces actress control over privacy simply by existing.
Without manual overrides, the requests sent to the vendors are mapped to the App Engine virtual machine instead of the user's browser or device.
There are no information leaks with tertiary-party cookies, there are no surprises with injected URL parameters, and the 3rd-political party service doesn't have any connection with the user's browser by default. They'll exist communicating simply with the cloud machine.
More command over HTTP traffic
To aggrandize the features mentioned to a higher place, yous will also have full command over what HTTP traffic is passed through the Server container.
Typically, the browser loads the vendor's JavaScript from their content distribution network (CDN).
This deed already exposes the user'due south browser or device to the tertiary party and tin can atomic number 82 to things like personally identifiable information (PII) leaks in case the URL of the page has sensitive information.
Because you at present accept a proxy between the device and the endpoint, the only place where this data is leaked is into your cloud environment.
Sure, it's still not optimal - PII leaks should be eradicated.
Only you have full control and buying of all the data collected by the Server container, and y'all also accept all the tools at your disposal to clean upwards and validate the payloads.
You can also cover your legal back by removing fingerprintable surfaces from the approachable requests from the Server container. Similarly, you lot tin can utilise the APIs available to hash potentially sensitive information. You can likewise, of course, look for consent strings in the user'southward cookies (assuming the Server container is in first-party context with the source of the traffic) and act accordingly.
Finally, in the Client, yous can modify the HTTP response back from the Server container to the browser or device. This is a pretty cool thing when considering Apple'due south Intelligent Tracking Prevention, for instance. Yous can convert cookies written with JavaScript, and thus subject area to an expiration limit of 7 days, into HTTP cookies written with a Set-Cookie
header, thus extending their lifetime to whatever you lot choose:
const getCookie = require('getCookieValues'); const setCookie = require('setCookie'); // Get cookie from the HTTP request allow ga = getCookie('_ga'); // If no cookie exists, generate a new Client ID ga = ga && ga.length ? ga[0] : generateClientId(); // Write the _ga cookie in the HTTP response setCookie('_ga', ga, { domain: 'motorcar', 'max-age': 63072000, path: '/', secure: true, sameSite: 'lax' });
Ideally, y'all'll desire to set cookies with the HttpOnly
flag. This prevents the web page from accessing the cookie with JavaScript (document.cookie
). By allowing cookie access only for the webserver receiving the asking, you lot're introducing a decent back-up measure out for mitigating cross-site scripting attacks and preventing cookie values from leaking into cross-site tracking schemes.
The reason we're not using HttpOnly
in the example above is because cross-domain linking is something Universal Analytics withal does client-side with JavaScript.
Note! Yous might want to read this commodity on FPID to see what Google is working on in terms of improving the security of the Google Analytics cookie.
In whatsoever case, using the Set-Cookie
header like this removes the need for complicated APIs to practice the cookie rewriting for you, as you lot tin can just piggy-back the cookie rewrite on the information collection itself.
Content Security Policy
Some other benefit of reducing the number of HTTP endpoints with which the browser communicates concerns your site's Content Security Policy (CSP). A CSP is what your site would apply to restrict the HTTP traffic to and from the user's browser.
For example, if you add a JavaScript library that loads its content from Facebook to a site with a CSP, you'll need to petition the developers to relax that CSP and then that Facebook's domains would be allowed to send and receive information from the user'southward browser.
Naturally, the more than you lot relax the CSP, the less efficient it becomes.
By reducing the number of HTTP endpoints the browser needs to communicate with (because you've replaced them with your own Server-side tagging endpoint), you're making the CSP more than robust equally a event.
Clean up and validate payloads
Even if yous've managed to articulate the HTTP traffic itself of all potentially harmful data, you might nevertheless be left with URL parameters that the vendor requires you lot to populate. Sometimes, often even, these parameters incorporate PII, and yous'll desire to figure out a way to get rid of it.
I've written a lot about PII purging, and my customTask solution should be useful if you lot're sending data from the browser directly to Google Analytics.
But now with a Server container, you tin can build a Client which parses all the asking headers and the body looking for PII (so not just those related to Universal Analytics) and and then proceeds to make clean it up or obfuscate it.
You tin besides use the Server container to validate and fix requests.
For example, if yous ship a not-numeric value every bit the Outcome Value of a Universal Analytics request, that outcome will be nerveless by Google Analytics but discarded at processing. In that location's no warning in the browser - these hits just disappear.
Y'all could fix this in a Client by looking specifically for a faulty Event Value and converting it into a number:
You can see how this could dramatically improve your information quality once you start building Clients specifically designed for cleaning up your information streams.
Full control and ownership of the data collected by the container
This has already been mentioned before in this article, merely a significant role of edifice a server-side surround is mapping a subdomain to the endpoint. When the HTTP endpoint is able to respond to requests using a subdomain that's part of the same domain hierarchy equally the website sending the requests, the website and the HTTP endpoint exist in same-site or starting time-party context. This has a significant impact on how browser tracking protections treat the traffic.
The custom domain should be mapped using A/AAAA DNS records rather than a CNAME alias. The latter is a less useful solution for cookie permanence due to browser tracking protections.
Other than the question of domain context, a very important attribute of ownership is what's promised by the platform you subscribe to (Google Cloud Platform at the time of release).
Y'all have full control and ownership of the data in your Google Cloud project. Aye - you lot demand to trust Google on this hope. Hither's what they guarantee in the relevant documentation:
-
Google Cloud merely processes data that you instruct it to process.
-
Y'all ain your data. No data is processed by Google Deject for advertising purposes.
-
You'll always be aware of where your data is regionally located.
-
Your information is secured by independently certified and audited security standards.
This is pretty significant. Since you lot own and control the data nerveless by the Server container, its usage and data processing falls under the privacy policies, T&Cs, and contracts your organization has with its customers and end-users.
If a data leak were to happen, for example, the first place it would "leak" to would exist a data store that you own, and you can mitigate the fallout past making sure these leaks do not extend to the third parties to which you send the data from the Server container.
Naturally, as before long as your Server container does fire tags that send the data to 3rd parties, you introduce additional data processors and controllers to the mix, just having the "buffer" of a data shop and processor that yous own in between should assist a great bargain in mitigating security/privacy risks and liabilities.
Limitless possibilities
By shifting the processing of information to your Server-side endpoint you introduce a fairly inexpensive, scalable, and multi-functional proxy for processing data before it is wrapped upwardly and sent to the vendors.
In add-on to the benefits listed above, in that location are then many things you could do with a server-side setup like this:
-
Just collect the bare essentials from the browser: content ID for content, session ID for session, and user ID for user. In the Server container, use other APIs and services at your disposal to enrich the data using these IDs as the keys.
-
Run expensive lawmaking in the scalable environment of the server rather than as a brunt on the user's device. Things similar IP lookups and cryptographic hashing could merely as well be done in the Server container.
-
Maybe at some betoken nosotros'll meet native integrations to other Google Cloud products.
-
Imagine beingness able to write directly to Google BigQuery from the Server container without having to worry about complicated authentication.
-
Imagine triggering Cloud Functions past using Pub/Sub rather than HTTP requests.
-
Imagine utilizing Cloud logging to build a real-time monitoring system for the integrity of your data pipeline.
-
In one case the platform matures and once the library of bachelor APIs and custom templates is extended, the accomplish of Server-side tagging is merely limited by the imagination of its users.
Key concerns
Moving tracking away from the device to behind the veil of the server doesn't come up without its concerns.
The paradigm shift that we tin envision with GTM's Server-side tagging isn't just one of improving data drove; it's also one of obfuscating it.
Circumvent content blocking
One of the start knee joint-jerk reactions many probably have to Server-side tagging has to do with content blocking and browser tracking protections in general.
A typical approach for privacy-friendly browsers is to restrict or downright cake communications between the browser and known trackers. The list of known trackers is normally based on a blocklist such as Disconnect.Me, but information technology could also be algorithmic and on-device, such as with Intelligent Tracking Prevention.
Indeed, an endpoint like google-analytics.com
could well be targeted past the heuristics used in content blockers, simply my-server-side.domain.com
probably isn't.
Can y'all utilize Server-side tagging to circumvent content blockers? Admittedly. Should you lot? Definitely not; at least if that's your primary reason.
Yet, this does raise an interesting paradox.
It's not your fault that content blockers exercise non target your domains.
You are not obliged to exhaustively exam if all the actual endpoints are blocked by the browser. That would be a huge waste of resources and counter-productive to what Server-side tagging kickoff and foremost does: reduce client load.
Once server-side proxies get the norm (with a popular tool similar Google Tag Director likely to spearhead the transition), content blockers volition accommodate their heuristics to not but look at domains but also the information that is being sent.
The right class of action is to exist transparent at what data is being collected on your site, placing behind consent that which is required by law, and giving opt-out mechanisms for the rest.
And if information technology just and then happens that your endpoint gets blocked by content blockers or its URL string is stripped of all useful information, don't try to "fix" this.
Always err on the side of maximum privacy.
Always assume that the user knows exactly what they are doing when they choose to cake your data collection. Don't defy their wishes.
Opaque data drove
When there'southward a data leak or a security breach in a company, the party uncovering this is often not related to the company at all.
The web is full of companies and individuals who exhaustively inspect the HTTP traffic in and out of websites, and their work has been instrumental in uncovering things like Magecart attacks and domain takeovers.
Cookie leaks, cantankerous-site scripting injections, CSP workarounds, and all style of nasty JavaScript hacks tin typically be audited direct in the browser, because the vendor scripts are running right there nether the watchful optics of the auditors.
When you lot move to Server-side tagging, you lot are reducing the corporeality of third-party JavaScript running on the site, which is practiced. It's a great step to mitigating the issues listed in a higher place.
However, you are also removing all traces of what is really done with the data bundled in the requests. Auditors will accept a hard time deciphering just what the event stream to your endpoint really does, and whether you are compromising the user's right to privacy and security behind the veil of the server, where client-side tools tin't reach information technology.
This ways that yous must document advisedly what type of data is being collected and processed on your site. Yous are already obliged to practise then under legal frameworks like GDPR and CCPA/CPRAA, which require you to be upfront and transparent about data drove, storage, and processing.
Always err on the side of maximum privacy.
You should take preemptive and proactive measures to exercise transparency and compliance right in lodge to avoid litigation and potential brand damage when you get defenseless in the act.
Consent direction is up to the admin
This isn't that far removed from what the situation is currently with customer-side scripts, just it's even so something you lot need to consider.
Consent management is a hot topic, and rightfully so. Many sites implement client-side consent direction tools, which require opt-in input from the user with regard to what data they allow to be collected from them.
Typically the consent string is stored in a cookie or localStorage
entry, and many vendors can really proactively react to consent frameworks such as IAB's Transparency & Consent Framework two.0.
When you move to Server-side tagging, you might accept just a single stream of events from the browser to the server. This single stream can be split into dozens and dozens of advert, marketing, and analytics requests in the Server container.
The templates running in the Server container won't exist able to leverage client-side consent APIs such as those suggested by TCF. Instead, you demand to build the mechanism of interpreting and parsing user consent manually in the container itself.
Peradventure, and hopefully, Google will introduce tools that brand this process easier. All the same, until then you need to make sure that when consent is nerveless in the browser or device, it is respected in the server too.
Toll
The cost of running a Server-side tagging is large or small, depending on what yous're comparing it to.
It'south large compared to just running the scripts in the browser, thus accumulating zilch extra cost.
It'southward small compared to all the benefits you'll arrive return for setting up the container, at to the lowest degree I like to call back then.
Running a three-instance App Engine setup for my site of small-scale traffic puts me back about 120€ per month. For me, this is acceptable because I get more than control over the data collection on my site.
Poor availability of server-side endpoints
For the Server container to piece of work equally a replacement for your browser- or app-based tracking, the vendors you lot desire to send data to demand to be able to collect the HTTP requests sent by the Server container.
This isn't necessarily a big event - vendors e'er take an HTTP endpoint to which their JavaScript library sends information, and many support simple image pixels for collecting the GET requests.
However, many vendors also stuff the browser with super complicated and heavy JavaScript. If you lot want to work towards reducing the amount of third-party crap loaded in the browser, the vendor should provide a ways to build the payload manually, without having to load their bloated JavaScript libraries.
For example, Facebook has a big and very complex set of client-side libraries they want you to download when working with their pixel. The purpose of the library is to let you communicate with Facebook'due south servers using the fbq()
command queue.
Luckily, Facebook also offers the Conversions API, which specifies a format for the HTTP asking with the conversion information. By deciphering the Conversions API documentation, anyone can build their own event stream from the device to the Server container to Facebook without having to load whatever Facebook JavaScript at all.
Then there are services like HotJar that are so tightly coupled with client-side interactions that it is unlikely you can always run HotJar without having to download their JavaScript. It will exist interesting to run into how vendors like HotJar arrange to a tagging environs that is run completely in the server.
Technical outline
When you lot follow the instructions to provision a Google Tag Director server-side surroundings, the scripts automatically create a unmarried Standard App Engine instance.
NOTE! In early beta, at that place was also the choice to load the Server container in a Kubernetes cluster of Compute Engine instances. This is a more than avant-garde setup designed for users who already have a pipeline running in Google Cloud, and they want more control over it than what App Engine's managed environments can offering.
App Engine is a managed virtual machine platform running in the Google Cloud. Past using the Standard environment and a single instance, y'all tin test-bulldoze your Google Tag Manager setup most probable without even expending the free quota you have available.
Yet, as before long equally you're ready to deploy the new Server container into a full production environment, you lot should guarantee best performance and uptime by transferring to a Flexible App Engine surroundings and increasing the number of instances to a minimum of three. Past doing this, you lot increase the throughput and performance of your server-side endpoint, and yous guarantee that information technology is able to shoulder the incoming load. Follow these instructions for more details.
In addition to provisioning extra instances, you should also map a custom domain to the endpoint, preferably one that is a subdomain of your main site.
Price
It'south difficult to say what the exact cost for your setup will be, only rest assured that there will be costs associated with product usage.
As an case, I'thousand running App Engine in a Flexible environment, using iii instances (the minimum recommended setup for production apply). The cost associated with this setup is around 4 euros per day.
I'm just collecting Universal Analytics Folio Views to this Server container. Every bit y'all can come across, the graph is pretty much steady regardless of the amount of hits coming in (my site averages just 0.2 requests per second).
My site has a visible dip in pageviews over weekends, with around 8,000 pageviews sent over a typical weekday and just ane fourth of that over a Sabbatum or a Sun. However, these dips don't reverberate in the cost of running my current Server container, which means I could probably calibration the setup downwards a little, but on the other mitt I fully intend to add additional measurements, and so I'd have to scale back up anyway.
When yous compare the cost forecast above with a Server-side tagging setup that collects around sixty requests per 2nd, we're talking at around 250€ per month instead.
I hope at some signal Google releases case studies and statistics, or even a tool, which let you to estimate the toll and scale up or down accordingly.
Server container
The Server container itself is visually reminiscent of any Google Tag Manager container.
The main divergence is the new Customer nugget blazon you tin see in the left-hand menu. I'll explicate more nigh clients in the associated affiliate.
When using the term incoming HTTP request, I'm referring to the HTTP asking that is sent from a device or browser to the Server container. When using the term outgoing HTTP request, I'm referring to the HTTP asking built and dispatched by tags firing in the container.
Tags
Tag-wise there'south not much there, nevertheless.
There's the native Universal Analytics and App + Web templates, both configured to assimilate information pushed by their respective Clients. The HTTP Request tag lets you create an approachable HTTP request to any destination.
And then there are all the custom tag templates people can imagine creating. Most any service that accepts HTTP requests can exist configured into a custom tag template in the Server container.
Triggers
In that location's a noticeable lack of available triggers. In fact, there's just a single Custom trigger type.
The fact is that a Server container would not be associated with arbitrary triggers such as "Folio View", "Click", or "Video". Instead, any tags triggering in a Server container would only trigger if a Client instructed them to do so.
A Server container is too unrelated to client-side labels such as a "folio load" or a "container load". It's running all the time - it'south non reset when the page is refreshed. Thus at that place are no triggers related to the lifecycle of a Server container, though that doesn't mean there won't be at some bespeak.
Variables
The available Built-in variables are:
Variable name | Description | Instance |
---|---|---|
Query String | Returns the query string of the incoming HTTP request. | v=1&t=pageview&tid=UA-12345-ane... |
Asking Method | Returns the method of the incoming HTTP request. | Go |
Asking Path | Returns the path of the incoming HTTP request. | /collect |
Client Proper noun | Returns the name of the Client currently processing the asking. | Facebook Client |
Container ID | Returns the ID of your Server container. | GTM-XXXXXX |
Container Version | Returns the current version of your Server container. | QUICK_PREVIEW |
Debug Style | Whether the container is in Preview style or non. | true |
Random Number | Returns a random positive integer. | 12345 |
Event Name | Returns the value of the event_name field in the event data object that was passed to the container. | page_view |
These can be used to parse information about the incoming asking and to think metadata near the event that fired and the container itself.
Available User-divers variables are:
Variable proper name | Description | Example |
---|---|---|
Cookie Value | Set up to the value of the offset cookie that matches the name. | GA1.2.12345.12345 |
Query Parameter | Set to the value of the first query parameter in the incoming HTTP request that matches the name. | UA-12345-i |
Query String | Returns the query cord of the incoming HTTP request. Note! Use the Built-in variable instead. | five=1&tid=UA-12345-1&t=pageview... |
Request Header | Returns the value(due south) of the header name from the incoming HTTP request. | https://referrer-page.com/ |
Request Method | Returns the method of the incoming HTTP request. Note! Use the Built-in variable instead. | POST |
Request Path | Returns the path of the incoming HTTP request. Note! Utilise the Built-in variable instead. | /j/collect |
Client Name | Returns the name of the Client currently processing the request. Annotation! Utilise the Built-in variable instead. | Universal Analytics |
Constant | Returns whatever cord you type into the variable. | UA-12345-1 |
Event Data | Returns the value of the central in the upshot data object. | 123.123.123.123 |
Outcome Name | Returns the value of the event_name field in the consequence data object that was passed to the container. Annotation! Utilise the Built-in variable instead. | page_view |
Random Number | Returns a random positive integer. Annotation! Use the Congenital-in variable instead. | 123123 |
Container ID | Returns the ID of your Server container. Notation! Use the Built-in variable instead. | GTM-ABCDE |
Container Version Number | Returns the electric current version of your Server container. Note! Use the Congenital-in variable instead. | xiii |
Debug Style | Whether the container is in Preview mode or not. Note! Use the Congenital-in variable instead. | false |
You can, and should, utilize custom templates to build your own variable types.
Custom domain
You are strongly encouraged to map a custom domain to your Server container endpoint.
The main reason is that this way you tin can incorporate the server-side information drove endpoint that y'all ain in your showtime-party domain namespace. For example, I'm using sgtm.simoahava.com
as the host of the Server container.
This becomes significant when you consider things like Intelligent Tracking Prevention. You might want to make employ of cookies in the incoming requests, but if the Server container is hosted on a domain that is different from where the requests are sent (such as your site), these cookies will be considered third-political party cookies and thus dropped by many browsers.
Notation! Due to upcoming changes in ITP, y'all should map the domain as a newly verified subdomain. This way you'll be instructed to use A/AAAA DNS records rather than the vulnerable CNAME alias.
Similarly, having the endpoint in your first-party domain namespace means y'all tin can do things like prepare first-political party cookies with Set-Cookie headers and thus avert Safari expiring them within seven days of being set.
Note that technically yous can have more than ane domain pointing to your App Engine deployment. This is helpful in instance you accept a single server-side container responding to requests from multiple domains. The only "catch" is that some features of the Server container, such as for which domain the Preview manner is shown, are restricted to just one domain. It won't hamper the actual data drove, but it might make it difficult to use some of these features.
Requests and responses
Server-side tagging revolves effectually incoming HTTP requests to the server container beingness mapped by a Client, passed to a tag, and then dispatched every bit an outgoing HTTP request to the tag vendor. In one case all the tags have fired for a given Customer, the Client sends an HTTP response back to the origin of the asking, such as a browser, an app, or some other connected service.
This flow is absolutely cardinal to understanding how Server-side tagging works. A "perfect" end-to-end pipeline would be ane where the requests are carefully sculpted to brand use of as footling customer-side code every bit possible; The Clients are designed to handle both vendor-specific and vendor-doubter requests; The tags are built to trigger off specific clients' event information payloads, finally responding to the Customer whether the approachable HTTP request was a success or not.
The easiest fashion to map a Client to an incoming request is to detect the Request Path. The congenital-in Universal Analytics Customer, for example, is primed to listen to requests that accept the /collect
path or any of its permutations (such as /j/collect
or /r/collect
). You could create a Facebook Client that listens for a custom path of /fbook
, and a HotJar Client that listens for requests with the path of /hotjar
.
Alternatively, yous could arroyo a unmarried event stream model, where all requests are sent to the same Client. In this example, they would have only a single path, such as /events
, and you would configure the Customer to parse the request body and turn information technology into event data objects that are passed to the container.
Whatever you choose, yous demand to remember that whatever origin sent the asking is actually waiting for a response dorsum from the Server container.
By default, the response is a adequately nondescript text/html
response, only you can jazz things up using the following APIs.
-
setCookie
lets you write cookies in the Set-Cookie header of the HTTP response. -
setPixelResponse
automatically configures the response to mimic a 1x1 GIF pixel with headers that foreclose caching. -
setResponseHeader
can be used to add any custom HTTP headers to the response. Consider headers likeAdmission-Control-Allow-Origin
andAdmission-Control-Permit-Credentials
, which are useful for preflight requests, for example.
Hopefully at some point we'll see more than options for manipulating the request every bit well. We could envision Clients whose only purpose is to purge the incoming request from PII, before passing the request on to the adjacent client that actually builds the effect object for tags to dispatch, now cleared of all PII.
Clients and tags
Nosotros've already talked a lot virtually Clients and tags, simply it's good to reiterate equally the concept might be a flake alien - especially if you lot're used to how GTM for the web works.
The purpose of a Client is to listen for incoming HTTP requests, parse them into a unified consequence model, so run a virtual container with the effect model, and then that tags can use the details in these issue objects to compile the requests to their endpoints.
Because Clients do all the legwork, you tin can start streamlining the consequence stream itself, moving away from vendor-specific request parsing (which might require vendor-specific JavaScript to run in the user'south browser, for example), and leaning towards a more agnostic approach, where a single event stream tin can exist distributed into multiple unrelated endpoints.
Clients operate on a Priority guild. The college the priority of a Customer, the sooner information technology gets to check if the request is its to claim. To merits a request means calling the claimRequest
API. This, in turn, ways that the electric current Client tells all the other Clients that this request is MINE! and doesn't allow any other Client to parse the request anymore.
Once the Client has parsed the asking and congenital an upshot object out of it, the event is passed to the runContainer
API. With this API, the consequence is passed to the tags to evaluate and potentially trigger with.
Tags tin exist set to trigger on numerous different things, but most probable you will end upwardly using a combination of Event Name and/or Client Proper name.
The Event Proper noun is something that gtag.js, Firebase Analytics, and more recently App + Web iterated and introduced to the earth of web analytics.
Basically, there's an inventory of suggested or quasi-standardized Upshot Names, such as page_view
, search
, and login
. Then there is always the opportunity to utilise a custom Event Proper noun such as new_level_achieved
.
When the Client builds the event model, it has to provide an Event Name. This is what ends up showing up in the Preview screen when the request is claimed and mapped by a Client:
So if you wanted to fire a agglomeration of tags whenever a page_view
is nerveless, regardless of Client, yous'd only utilize a trigger that checks the Upshot Name. Y'all'd then just demand to assume that the Client has correctly intercepted the incoming HTTP asking, and has managed to parse and map it into an effect object that tin can be understood by your tag (y'all can use Preview mode to analyze what the issue information object contained).
Alternatively, perhaps yous desire your tag to fire only when the Facebook Customer generated the event object. This is useful in case the tag requires a lot of custom information not available as standard parameters in the event object.
By referencing the Client Name, y'all can ensure that the tag only fires if the correct Client has claimed the incoming HTTP request, assuming the Client that executed runContainer
too claimed the request (there might be some edge cases where this is not the case).
Information technology might be hard to wrap your caput around Clients and tags, but once the inventory of templates for both multiplies in the community gallery, it will go easier to simply apply the community templates rather than having to worry about how to build your ain.
Effect model
When the Client parses an incoming HTTP request information technology has claimed, it needs to map values in the request torso (typically in a query string) and produce an event data object, which looks something like this:
{ event_name: "page_view", event_parameter: "some value", event_parameter_object: { nested_event_parameter: "some other value" } }
This object is what gets passed to the container with the runContainer
API. Tags will and then be able to use these values in the trigger, e.g. firing a tag only when event_name
is page_view
, and they'll be able to map values in the event object to the approachable HTTP request they dispatch to the vendor endpoint.
To keep things streamlined, Google suggests a ready of standard result names and parameters that you lot should endeavor to follow to make certain that the effect data object passed to the container tin can be used with as piddling friction every bit possible. If the tags require parameters that are not available in the listing of standard parameters, they should be prefixed with x-vendor-
. Thus, for example, Facebook's Subscription ID becomes x-fb-subscription_id
.
Note besides the preference of snake_case
vs. camelCase
. It's a syntactical format you lot should get accustomed to using when working with Server-side tagging.
You tin can always use the Server container'southward Preview mode to audit what'southward passed in the event data object by whatsoever given client. For example, when collecting a Universal Analytics Measurement Protocol hitting, this is what you might run across:
In the instance above, many standard parameters are populated, such every bit client_id
, ip_override
, page_location
, and user_agent
.
Additionally, Measurement Protocol uses a number of custom parameters that are (more or less) unique to Google Analytics, such every bit x-ga-mp1-vp
(viewport size), 10-ga-measurement_id
(spider web holding ID with Universal Analytics), and x-ga-mp1-plt
(page load time).
This result object is then digested by the Universal Analytics tag, which will be able to take these items and compile the outgoing Measurement Protocol asking to Google Analytics. If the event object is correctly compiled, the tag can even utilize the shorthand API sendEventToGoogleAnalytics
.
Custom templates
Server-side tagging relies heavily on custom templates. In addition to tag and variable templates, which are available in web containers also, ability users now have the opportunity to create Customer templates as well.
The available APIs for these are listed in the documentation. Many APIs, such equally runContainer
accept a footnote saying:
It is recommended that this API be used from a client template.
Roughly, Clients should typically utilise APIs that claim, validate, and parse the incoming HTTP requests, run the container with the event data object, listen for messages from the tags fired in the container, and finally modify and return a response back to the source of the incoming request.
Here are some APIs that you'd typically run exclusively from a Client:
Client API | Description |
---|---|
claimRequest | Merits the asking for the Client. |
extractEventsFromMpv1 | Parse an incoming Measurement Protocol v1 request, and extract event data objects from it. |
extractEventsFromMpv2 | Parse an incoming Measurement Protocol v2 asking, and extract issue data objects from information technology. |
getCookieValues | Get the values of all cookies with the given proper noun in the incoming HTTP asking. |
getRemoteAddress | Best-effort effort to get the IP address of the incoming asking. |
getRequest* | All the getRequest... APIs are designed to parse some attribute of the incoming HTTP request. |
isRequestMpv1 | Check if the incoming HTTP asking is in Measurement Protocol v1 format. |
isRequestMpv2 | Check if the incoming HTTP request is in Measurement Protocol v2 format. |
returnResponse | Return the HTTP response with all the set headers dorsum to the source of the incoming request. |
runContainer | Run the container with the outcome data object. |
setCookie | Populate a Set-Cookie header in the response. |
setPixelResponse | Automatically prepare response headers to mimic a 1x1 GIF pixel. |
setResponse* | All the setResponse... headers modify some aspect of the HTTP response finally flushed by the returnResponse API. |
Note that you certainly can utilise some of these APIs in tags. For case, you could set aspects of the response directly in the tag itself. Nonetheless, it might brand sense to take tags communicate their condition back to the Customer with the
sendMessage
API, and use the Client to manage all aspects of the request-response catamenia.
Tags should typically use APIs that parse the event data object, build an HTTP request to the tag endpoint, and message dorsum to the container whatever metadata the approachable asking produced (such every bit a failed status lawmaking or success message).
Here are some APIs that y'all'd typically run exclusively from a Tag:
Tag API | Clarification |
---|---|
getAllEventData | Get the result information object passed to the container. |
getClientName | Become the name of the current Client. |
getEventData | Get the value of a single cardinal in the event data object. |
sendEventToGoogleAnalytics | Automatically build and dispatch an outgoing Measurement Protocol request from an event data object formatted correctly. |
sendHttpGet | Transport an HTTP GET request. |
sendHttpRequest | Send a fully customizable HTTP request. |
Note that y'all could run a Server container without a unmarried tag. All the APIs designed to be used in tags could be run through a Client. Simply this is orthogonal to how Server-side tagging has been designed to piece of work.
Here'due south an case of a Client running some of the recommended APIs:
const addMessageListener = crave('addMessageListener'); const claimRequest = require('claimRequest'); const extractEventsFromMpv1 = crave('extractEventsFromMpv1'); const isRequestMpv1 = require('isRequestMpv1'); const returnResponse = require('returnResponse'); const runContainer = require('runContainer'); const setResponseBody = crave('setResponseBody'); const setResponseHeader = require('setResponseHeader'); const setResponseStatus = require('setResponseStatus'); const setPixelResponse = require('setPixelResponse'); // If Measurement Protocol request, claim and parse if (isRequestMpv1()) { claimRequest(); const events = extractEventsFromMpv1(); // Listen for bulletin from tag signalling completion, // gear up response headers accordingly. addMessageListener('ga_complete', (messageType, message) => { if (message.status === 'error') { setResponseStatus(500); setResponseBody(message.body); } else if (message.condition === 'redirect') { setResponseStatus(302); setResponseHeader('location', message.location); }else { setPixelResponse(); } }); // Run the container with the parsed result object let eventsCompleted = 0; events.forEach(effect => { runContainer(issue, () => { // If all the events in the incoming HTTP request take been completed if (events.length === ++eventsCompleted) { returnResponse(); } }); }); }
And here's what the corresponding tag might exercise:
const getAllEventData = require('getAllEventData'); const sendEventToGoogleAnalytics = crave('sendEventToGoogleAnalytics'); const sendMessage = crave('sendMessage'); // Access the outcome data object const consequence = getAllEventData(); // Ship the event to Google Analytics and parse the response sendEventToGoogleAnalytics(event, response => { if (!response.success) { sendMessage('ga_complete', { status: 'error', body: 'Request to Google Analytics failed' }); render data.gtmOnFailure(); } if (response.location) { sendMessage('ga_complete', { status: 'redirect', location: response.location }); } else { sendMessage('ga_complete', { status: 'success' }); } information.gtmOnSuccess(); });
As you tin can run into, the Client parses the incoming request and sends it every bit an event data object to the container. The container then triggers the tag(s), which map the event data object to a Measurement Protocol request.
Google Analytics is a scrap special in this case, considering both the incoming request parser (extractEventsFromMpv1
) and the outgoing asking dispatcher (sendEventToGoogleAnalytics
) have their ain, dedicated APIs congenital. If you use a custom vendor endpoint, you lot demand to actually manually write the code that turns an incoming request query cord into the event data object, and which takes the outcome data object and maps it to an outgoing HTTP request.
The addMessageListener
and sendMessage
APIs are very useful, as they allow tags and Clients to communicate with each other. This is very helpful, in example y'all want the Client to encode information most tag execution in the response back to the asking source.
Note! The sample code in a higher place is a bit unwieldy, because information technology assumes
runContainer
to only trigger i tag. If there's more than 1 tag firing, theaddMessageListener
callback would react to each tag messaging dorsum, which means only the message from the last tag that fired would be considered for the response the Client sends back to the source of the incoming request.
Preview and debug
The Server container, just like a spider web container, has its own Preview and Debug style. When you click the Preview push in the user interface, a new tab opens with the Preview interface.
But like with a web container, the Preview tab only shows hits that originate from your browser - and it has to be the same browser instance that started Preview mode.
In the left-manus side navigation, you lot see all the incoming HTTP requests and whether or non a Customer has claimed and created an issue data object from the request. In the screenshot, no event data object was created for the favicon.ico
request, but both collect
requests were claimed by a Client and turned into event information objects.
If you select a asking, all the tabs in Preview style will behave equally if y'all'd selected Summary. In other words, you'll be able to meet how many times a tag would have fired for the request, but you wouldn't be able to wait into Variables or Event Information.
For this reason, whenever you lot debug, you lot should cull the upshot data object (e.g. page_view
in the instance) if available.
Request tab
The Request tab contains information about the incoming HTTP request and the response sent back past the Server container. It will also evidence you if a Client claimed the request.
The very first line contains a summary of the HTTP request itself. Afterward that is the value of the event_name
field, parsed by the Client from the incoming HTTP asking.
The Client box tells you which Client claimed the asking.
The Incoming HTTP Request box opens up a new overlay when clicked, with details about the request.
The Request overview shows you the Asking method (e.g. Get
or Postal service
), and the URL to which the request was sent.
Request Headers list all the HTTP headers nowadays in the asking, and the Asking Body carte du jour shows what was sent equally the body of the request (typically merely in Mail
requests).
The Response overview has details about what the Server container responded with back to whatever source sent the incoming HTTP request in the first place.
Status Code indicates whether or non the request was a success (200
).
Response Headers typically include things similar enshroud headers for preventing the browser from caching the request, and Ready-Cookie
headers which write a cookie on the domain running the Server container.
If the response has a trunk, it's displayed in the Response Body card.
Tags tab
The Tags tab is pretty self-explanatory. You'll see all the tags that fired (or did not fire) with this upshot information object. Similar to a web container, you can click open a tag to meet what values information technology sent, and you can scroll downwards to its triggers to come across why information technology didn't burn.
One cool addition is the Approachable HTTP Requests box. When you click it, a new overlay opens with details about the HTTP requests sent by the tag.
You lot can use this information to debug whether or non the hit to the vendor was dispatched correctly.
Variables tab
The Variables tab, similarly, tells you what the value of each configured variable in the container was when the container was executed with the effect data object.
This is a useful list of data, as you tin can use it to debug why a tag might not have sent the correct values to the endpoint.
Event Information tab
This is a very important tab to familiarize yourself with because it shows you all the values parsed from the incoming HTTP request into an event data object. Yous can employ this with the Asking tab to run across which parameters might be missing or which were parsed incorrectly.
Errors tab
If any tag throws an error, this tab would have more than information almost information technology. Y'all can read more than about the errors tab.
Resources
Hopefully, this article will serve as a solid resource for you, particularly when you lot're getting your feet wet with Server-side tagging.
In addition to this, I'd like to direct you to the official documentation:
- Overview and deployment instructions
- Provision boosted servers (minimum of iii is recommended for production)
- Setup a custom domain
- API documentation for templates
- Suggested
event_name
values and parameters in the event data object
Summary
I fully expect Google Tag Manager's Server-side tagging to change the landscape of digital analytics. They're not the first vendor to innovate a handy mode of building server-side proxies, merely they're Google, and this service doesn't come with a license cost.
There are obviously many concerns well-nigh Server-side tagging. Moving tracking backside the veil of the server will most certainly bristle some hairs. For this reason, I recommend you strive for absolute transparency when disclosing details near what tracking is going on either directly or indirectly in your digital assets and properties.
I'm certain we'll meet a proliferation of new custom templates introduced specifically for the Server container, and I also expect nearly vendors in the adtech and martech infinite to facilitate server-to-server communication for their pixels and data collection endpoints.
Please let me and other readers know in the comments what you retrieve of Server-side tagging. Do permit me know if there are things in this article that require clarification.
Source: https://www.simoahava.com/analytics/server-side-tagging-google-tag-manager/
0 Response to "Crm Server Side - Unable to Read Data From Transport"
Post a Comment