Frequently asked questions

   General

   Problems

   Features

(Note: Due to excessive spam, we have turned off public editing of this page. If you would like something added, please email us.)


General

Do I need to install anything to use CoralCDN?

No. You merely need to append .nyud.net to the hostname of the URL you wish to access via CoralCDN. For example, witness the Coralization of the following URL:

http://www.example.com/info/ becomes http://www.example.com.nyud.net/info/

Because you are just accessing URLs in your browser, Coral can be used with virtually any operating system (Windows, Mac, Linux, BSD, etc.) and web browser (Mozilla, IE, Netscape, Opera, etc.).

What's a Coralized URL?

It is just our name for URLs of the form something.nyud.net, where something is the hostname of the origin server. Beyond the adjective form, it can also be used as a verb: e.g., to Coralize a URL. Or a noun, as above. What a fun addition to the English language!

Why should I use CoralCDN?

As a web-site operator, you can greatly reduce your bandwidth usage by redirecting clients to CoralCDN, as well as providing better performance when your web server would otherwise be overloaded.

As an interested third-party -- such as the poster to a popular Web portal or mailing list -- you can ensure that your readers can still access a certain web page or files, when the multitude of readers would otherwise overload the website and make the content unavailable.

As a user, CoralCDN can help you reach content that may be efficiently cached nearby. If several nearby users access the same web pages, the can enjoy better performance by leveraging Coral's cooperative cache, rather than using the possibly-distant origin web server. Try installing a browser plugin.

For more information on how to use CoralCDN, see our usage page.

Can CoralCDN access web servers running on ports other than 80?

Yes. If example.com's server was instead running on port 8080 (it isn't), you could access it via the Coralized URL:

http://www.example.com:8080/info/ becomes http://www.example.com.8080.nyud.net/info/

Can I run a CoralCDN node?

Ultimately, we certainly want and hope that many third parties run Coral nodes, so that Coral can grow into a world-wide network of thousands of computers. For now, although the source is available via anonymous CVS, we'd prefer to run a network of several hundred machines on PlanetLab that are under our control, to enable easier maintenance, debugging, and pushing our regular changes, bug-fixes, and new functionality. However, feel free to use Coral regularly! In fact, we welcome your help and feedback as users.

Furthermore, there are more serious security issues we will have to handle once Coral is run on untrusted clients (one can think of the current deployment as "trusted", similar to commercial CDNs.) Until better security protections are in place, we want to retain control over Coral nodes.

Can I use Coral servers as upstream HTTP proxies?

CoralCDN is not designed to be used as a client-side HTTP proxy, although our browser plugins enable straight-forward use of CoralCDN by clients to access particular URLs.

Problems

Please stop Coral from probing my machine!

Coral only actively probes an IP address (using either a fast traceroute-like tool via ICMP ECHOs or a single TPC/ICMP ping message) after receiving a request from that same address.

Thus, if you notice that you receive some type of probe packets from Coral, this means that the machine which Coral is probing is either acting as a DNS resolver (and that some user of this resolver has recently attempted to access some .nyud.net address) or that the machine is running a web client which is attempting to access the Coral network (again via some .nyud.net address).

We do not port scan random IP addresses or ranges: Coral only probes IP addresses which contacted it first! Unfortunately, many intrusion detection systems cannot differentiate this, much to our dismay.

Coral does not seem to work for me

How sad! This can be due to a number of reasons. One big reason may be that you are behind some firewall blocking port 8080, or that you are using a Windows DNS server, which appears to be buggy.

First, check if DNS redirection is actually working via the nslookup or dig command (or what else might be available to you as a user). If you don't have access to such a tool, just try to use the address www.nyud.net in your browser. If this resolves to some PlanetLab web page, DNS resolution it at least working for you.

If you have access to nslookup or dig, you should get back an address (an A record) to some webproxy to use:

      % nslookup x.y.z.nyud.net

      Non-authoritative answer:
      nyud.net	dname = http.l2.l1.l0.nyucd.net.
      x.y.z.nyud.net	canonical name = x.y.z.http.l2.l1.l0.nyucd.net.
      x.y.z.http.l2.l1.l0.nyucd.net  canonical name = http.l2.l1.l0.nyucd.net.
      Name:    http.l2.l1.l0.nyucd.net
      Address: 216.165.109.81
      Name:    http.l2.l1.l0.nyucd.net
      Address: 216.165.109.82


      % dig x.y.z.nyud.net

      ...
      ;; QUESTION SECTION:
      ;x.y.z.nyud.net.			IN	A

      ;; ANSWER SECTION:
      nyud.net.		    14400    IN   DNAME  http.l2.l1.l0.nyucd.net.
      x.y.z.nyud.net.		0    IN	  CNAME  x.y.z.http.l2.l1.l0.nyucd.net.
      x.y.z.http.l2.l1.l0.nyucd.net. 3600 IN CNAME  http.l2.l1.l0.nyucd.net.
      http.l2.l1.l0.nyucd.net.    60    IN   A      216.165.109.81
      http.l2.l1.l0.nyucd.net.    60    IN   A      216.165.109.82

      ;; AUTHORITY SECTION:
      l2.l1.l0.nyucd.net.	     3600    IN   NS        216.165.109.82.ip4.nyucd.net.
      l2.l1.l0.nyucd.net.	     3600    IN   NS       216.165.109.81.ip4.nyucd.net.

      ;; ADDITIONAL SECTION:
      216.165.109.82.ip4.nyucd.net. 81876 IN A	 216.165.109.82
      216.165.109.81.ip4.nyucd.net. 81876 IN A	 216.165.109.81
      ...

But DNS returns SERVFAIL? See below.

Next thing to check is whether web proxies are either running on these machines (although CoralCDN attempts to return only recently-seen nodes), and that you might not be behind some firewall that blocks TCP requests to port 8080 or 8090, where CoralCDN is running.

      % telnet 216.165.109.82 8080
      Trying 216.165.109.82...
      Connected to 216.165.109.82.
      Escape character is '^]'.
      ^C

      Connection closed by foreign host.

You might be able to differentiate between the two error conditions, as if a Coral proxy is not running on port 8080 (and nothing else is), it will automatically just send you a RST packet (Connection Refused). If a firewall is blocking you, it might send you a RST or, probably more likely, it might just silently drop your request.

If you notice that Coral is working, but then you suddenly start getting connection refused messages, this may result when the web proxy you are using goes down. Coral specifically uses short time-to-live (TTL) for A records in DNS (the web proxy address) to reduce the impact of failures. Unfortunately, many browsers cache these mappings for much longer than the specified TTL (as a "performance" optimization, as the system-call API does not give applications the TTL). Thus, individual users may experience loss of performance for your browser's cache period, although other users that share the same DNS resolver will not see similar failures as they can receive fresh records from the resolver.

DNS does not seem to work.

I'm getting errors in my DNS syslog.

Some DNS servers do not support DNAME records (RFC 2672). Coral uses such records to move all requests for URLs of the form *.nyud.net to one of our web proxies. Once a local DNS resolver caches a DNAME record from a request to say, foo.nyud.net, when it attempts to access bar.nyud.net, it can reuse previously cached nameserver information and contact any of our 100s of Coral nameservers, without needing to contact one of our 10-13 "primary" nameservers. Without this type of caching, our primary nameservers would get a new request per hostname per resolver per day.

Unfortunately, Windows DNS servers have a long-standing bug, which prevents them from being "forward compatible". If DNS resolvers receive a response with a mix of resource record types, some of with which they are familar and some not, they should ignore the unknown record types and accept the known types. Unfortunately, Windows servers reject the entire packet if it includes a DNAME record (type 39) and write an error to syslog. To our knowledge, this long-standing, known bug applies to both Windows 2000 and 2003 servers. Note that most ISPs and companies, even if they use Windows desktop machines, usually use a UNIX implementation of DNS like BIND.

To partially support Windows servers, we currently return DNAME records probabilistically (33%). Thus, if a Windows server received a DNAME record only 1/3 of the times, it will only reject 1/3 of the packets. While this admittedly degrades performance, they usually at least re-request the domain name several times if need be, causing a request to succeed. A server that can handle DNAME records, like BIND, will cache the DNAME record for 24 hours once it receives the first packet with such a record.

Unfortunately, some Windows DNS servers are then configured to recursively forward requests to upstream servers running, e.g., BIND. In this case, once BIND caches the DNAME record, it will return the record to the Windows server, making all subsequent requests to the Windows server fail.

Therefore, we currently support

       client --> BIND    --> Coral
       client --> Windows --> Coral

but not

       client --> Windows --> BIND --> Coral

Users in that situation can try to append http.l2.l1.l0.nyucd.net (note the "C" in nyuCd) to the end of hostnames, instead of the traditional .nyud.net, to be able to use Coral today. Using the previous example:

http://www.example.com.http.l2.l1.l0.nyucd.net/info/

While this isn't a "pretty" solution, it may be particularly acceptable for those servers using Coral in a more surreptitious manner, e.g., by including Coralized URLs in RSS feeds. Note that using this alternative name does not affect Coral's behavior or load-balancing in any way.

(N.B. We have recently heard that some DSL routers with built-in DNS nameservers in firmware may have similar problems.)

Why don't you use port 80?

The use of port 8080 is largely a relic of our beta deployment on PlanetLab, given that it is a shared test-bed. We hope to switch to using port 80 as soon as possible. Sorry for any inconvenience; in the short-term, if you can't access port 8080 due to a local firewall and still wish to use Coral, try finding an open web proxy running on port 80. For example, Google for the search term "cgiproxy start".

As of July 2007, we now run on port 80 through port sharing on PlanetLab! Thus, you can now access CoralCDN through ports 80, 8080, and 8090.

When I access some personalized site via Coral, I no longer remain "logged in" with my preferences?

When you access www.example.com, the site .example.com can set a cookie for all of its subdomains (including www). However, when you access www.example.com.nyud.net, Coral, via the .nyud.net domain, cannot access any of your cookies for the .example.com domain. This security requirement ensures that sites cannot read potentially-private cookies set and managed by other domains. In fact, Coral doesn't support cookies at all.

Thus, Coral is not compatible with personalized content. Of course, as Coral is designed for static content, perhaps one should not necessarily be accessing personalized content via Coral.

The Coralized page does not appear to be fresh!

Coral is primarily designed as a caching network for static content, in order to reduce server load. That said, it currently sets a lower-bound on cache revalidation to 5 minutes. This means that, even if the no-cache HTTP directive is specified, it will only revalidate the content to the server (in order to reduce server load) once per 5 minutes. If you do not set any expiry period, Coral defaults to a 12 hour expiry period.

Note that because Coral does not store or forward cookies (as it cannot read pre-set cookies, as explained above), this behavior does not impact the privacy implications of ignoring no-cache and private directives.

Coral is not saving me bandwidth for my large file!

Because of bandwidth overuse, we temporarily capped off Coral to disallow transfers of files greater than 50 MB. Our current deployment has servers with 4 GB performing whole-file caching (and are running at cache capacity). If clients are pulling files on the orders of 100s of MB, the benefit of the system for hundreds of other websites is greatly reduced, as data would otherwise be quickly evicted from caches.

We're going to look at better techniques to do striped large-file caching to be able to handle larger files, but until we implement such functionality, large file transfers were otherwise killing our file caches.

Thus, instead of just returned some type of error message (like 403: Forbidden), we are transparently redirecting clients back to the origin site, where they at least have a possibility of downloading the file, and the server is not in worse shape than pre-Coral.

Coral is returning a 403: hourly quota exceeded message.

We have recently deployed some bandwidth capping algorithms on Coral, to allow individual servers to specify their maximum peak and steady-state bandwidth usage. Because of this, Coral may return a 403 FORBIDDEN message that says basically "Hourly quota exceeded; try back later" on a per-node basis.

This decision is calculated via three parameters: (a) a node's maximum peak hourly total bandwidth, (b) a node's desired steady-state hourly total bandwidth, (c) a node's steady-state bandwidth limit per hostname.

If a node is well-under its steady-state bandwidth (b), then the per-hostname limit (c) will dominate; if many different sites are competing for a node's resources, then the total hourly bandwidth (b) will dominate, and the algorithms will dynamically reduce the per-hostname limits. We calculate steady-state usage via an exponentially-weighted moving average, to enable flash crowds to take advantage of Coral while still capping use by long-term bandwidth hogs.

These algorithms thus allow greater fairness between multiple sites competing for Coral's resources. The current settings and deployment should allow a site to use Coral to distribute an upper-bound of 250 GB per day.

In the long term, having larger numbers of individuals running Coral nodes will increase the total available bandwidth and improved algorithms should take bandwidth limits into consideration when redirecting clients to proxies.

Features

What information do you log?

All requests to Coral are logged, mostly to aid in performance debugging and to identify abuse. For DNS, this includes the IP addresses of the client DNS resolver, timing measurements to the client, and the Coral nameservers and proxies to which the DNS query is resolved. For HTTP, this includes the client and server IP addresses, timing information, the URL requested, and the HTTP-Referer field.

In the case of abuse or suspicious traffic, we may access log information and may occasionally release entries to aid in investigations. While we also use logs in order to improve Coral's performance, we do not expect to intentionally release any personally-identifying information under normal operating conditions.

Why don't you automatically rewrite pages to Coralize URLs?

Coral does not provide this functionality largely due to its security implication. As we move from "trusted" nodes running on PlanetLab (under our control) to anybody running "untrusted" nodes, we need to concern ourselves that nodes may (maliciously) modify content. Thus, you may receive advertisements for prescription medicines or mortages or other less savory content, when in fact you are trying to access a completely different page, like CNN.

To handle this issue, we've written an apache module that servers can use to sign content to ensure content integrity and freshness, to be verified both and proxies and possibly by client browser extensions. (And it actually still supports streaming content through proxies, as we do now, by taking hashes over ranges of files for verification.) As Coral moves into the mode of anybody-running-a-node, we'll incorporate this feature to allow interested servers to gain that additional security protection.

If that's the case, Coral nodes cannot modify content, if only for the purpose of Coralization, as this would otherwise prevent us from using digital signatures to verify the authenticity of content, which are necessary to prevent the malicious modification of content by misbehaving nodes.

Does Coral support max-age, expires, and other HTTP cache-control directives?

Yes. Coral obeys HTTP/1.1 cache directives, with the exception of placing a minimum expiry time (using max-age) of 5 minutes, as described above. Thus, server operations can ensure that content accessed via Coral remains fresh using regular HTTP cache directives.

Does CoralCDN comply with the Digital Millennium Copyright Act?

Yes. It is our intent to comply with the Digital Millennium Copyright Act (the text of which can be found at the U.S. Copyright Office Web Site here) and other applicable intellectual property laws, through CoralCDN's technical design.

CoralCDN does not provide archival storage of content, like google.com's cache or archive.org. Much like a web cache or "content accelerator" at ISPs, CoralCDN only keeps data temporarily in its file caches, either until the data expires or it is evicted (as may occur for unpopular data). As described above, CoralCDN will serve data for some maximum fixed period (24 hours) before checking back with the origin website. If the content at that site has changed, CoralCDN will fetch the new content afresh, replacing the old. If the origin site is no longer online or the particular content returns some HTTP error message, CoralCDN will only serve the old data for a short time (24 hours).

Thus, if you believe that a website is making infringing content available, please direct any notices to that particular website. Recall that CoralCDN's naming method makes it obvious the identity of the origin website: A Coralized URL of the form http://www.example.com.nyud.net/ corresponds to an origin site http://www.example.com/ .

If/When that origin site complies with the notice, the content in question will naturally be removed from CoralCDN's caches through purely automated technical means in at most 24 hours.


Please note that these answers correspond to the best knowledge of CoralCDN's developers, and they do not constitute any guarantee of service or behavior. These practices may change at any time without notice.