Showing posts with label PostGres. Show all posts
Showing posts with label PostGres. Show all posts

Tuesday, December 4, 2012

Solaris 10: Using Postgres (Part 2)

(Postgres Logo)
Abstract:
Solaris had long been the operating system for performing managed services in the telecommunication arena. During a time when Oracle priced Solaris out of the market by charging a higher fee for similarly performing hardware than other competitors, Sun Microsystems started bundling Postgres and later purchased MySQL for bundling. Postgres is a simple, easy to enable, royalty free database available for Solaris. This article will discuss using the Solaris 10 bundled Postgres database.

Setting Up Postgres
The first article in this series, Solaris 10: Using Postgres (Part 1), discusses how to enable a reasonable 64 bit Posrgres database bundled with Solaris 10, preparing the first user, as well as running the first command line access.

Creating a Table



References
[html] Creating a Table
[html] Populating a Database Notes
[html] SQL Copy Data Into or From Table
[html] SQL Insert Into Table Command
[html] DML Inserting Data Into a Table
[html] SQL Update Data in Table
[html] SQL Delete From Table Command
[html] SQL Truncate Data in Table
[html] Insert or Update PG-SQL Expand 38-1
[html] Database Maintenance Through Vacuum

Monday, December 3, 2012

Solaris 10: Using Postgres (Part 1)

(Postgres Logo)
Abstract:
Solaris had long been the operating system for performing managed services in the telecommunication arena. During a time when Oracle priced Solaris out of the market by charging a higher fee for similarly performing hardware than other competitors, Sun Microsystems started bundling Postgres and later purchased MySQL for bundling. Postgres is a simple, easy to enable, royalty free database available for Solaris. This article will discuss setting up the Solaris 10 bundled Postgres database.
(Sun Microsystems Logo)

History:
From the first pages of the PostgreSQL documentation:
The object-relational database management system now known as PostgreSQL is derived from the POSTGRES package written at the University of California at Berkeley. With over a decade of development behind it, PostgreSQL is now the most advanced open-source database available anywhere. The POSTGRES project, led by Professor Michael Stonebraker, was sponsored by the Defense Advanced Research Projects Agency (DARPA), the Army Research Office (ARO), the National Science Foundation (NSF), and ESL, Inc. The implementation of POSTGRES began in 1986.
Postgres has existed a long time, from the same roots as Berkeley UNIX, the original base operating system for Sun Microsystem's Solaris.
(Sun Solaris Logo)

Versions:
Under Solaris 10, Sun Microsystems bundled Postgres. Basic directory structures are as follows:
V240/root$ ls -la /*r/postgres
/usr/postgres:
total 12
drwxr-xr-x  6 root bin  512 Jan  2  2010 .
drwxr-xr-x 44 root sys 1024 Mar  6  2010 ..
drwxr-xr-x 10 root bin  512 Jan  2  2010 8.2
drwxr-xr-x  9 root bin  512 Jan  2  2010 8.3
drwxr-xr-x  2 root bin  512 Jan  2  2010 jdbc
drwxr-xr-x  6 root bin  512 Jan  2  2010 upgrade

/var/postgres:
total 8
drwxr-xr-x  4 postgres postgres  512 Jan  2 2010 .
drwxr-xr-x 51 root     sys      1024 Nov 10 2010 ..
drwxr-xr-x  4 postgres postgres  512 Jan  2 2010 8.2
drwxr-xr-x  5 postgres postgres  512 Jan  2 2010 8.3 
Under Solaris 10, Postgres 8.2 and 8.3 are shipped. With 8.3, both 32 and 64 bit versions.

(It should be noted that with Postgres 8.3, community support is projected to end in 2012.)

File System Locations:
Before using Postgres, it may be advisable to mount additional disks in a ZFS pool and mount them. This is not strictly the "correct" way to set up a set of database directories, but for a small system where root disks are mirrored and a second set of mirrored disks are used for applications, it will be adequate.

V240/root$ zfs create zpool1/pg_8_3_backups
V240/root$ zfs create zpool1/pg_8_3_data  
V240/root$ zfs create zpool1/pg_8_3_data_64

V240/root$ zfs set mountpoint=/var/postgres/8.3/backups zpool1/pg_8_3_backups
V240/root$ zfs set mountpoint=/var/postgres/8.3/data zpool1/pg_8_3_data     
V240/root$ zfs set mountpoint=/var/postgres/8.3/data_64 zpool1/pg_8_3_data_64
 

V240/root$ zfs list
NAME                   USED AVAIL REFER MOUNTPOINT
zpool1                1.92G 65.0G 1.92G /u001
zpool1/pg_8_3_backups   21K 65.0G   21K /var/postgres/8.3/backups
zpool1/pg_8_3_data      21K 65.0G   21K /var/postgres/8.3/data
zpool1/pg_8_3_data_64   21K 65.0G   21K /var/postgres/8.3/data_64

V240/root$ cd /var/postgres/8.3
V240/root$ chown -R postgres:postgres *

The final 2 steps are critical, if ZFS file systems will be mounted and used, the default ownership is "root" and the starting process will fail if those ZFS directories are not owned by the dba "postgres".

When starting 8.3 version of Postgres, the data should now be stored on zpool1 application pool.

Services:
Postgres is a first-class citizen under Solaris 10. There are no start/sop scripts needed to be written - they are pre-bundled as a variety of services within Solaris Service Management Facility (SMF).
V240/root$ svcs "*postgres*"  
STATE    STIME    FMRI
disabled 12:49:12 svc:/application/database/postgresql:version_82
disabled 12:49:12 svc:/application/database/postgresql:version_82_64bit
disabled 12:49:12 svc:/application/database/postgresql_83:default_32bit
disabled 12:49:12 svc:/application/database/postgresql:version_81
disabled 12:49:13 svc:/application/database/postgresql_83:default_64bit
 The database version of choice can be enabled through SMF.

Review Database Owner:
Solaris comes with role based access to Postgres pre-installed. They should look similar to the following:
V240/root$ grep postgres /etc/passwd /etc/user_attr /etc/security/exec_attr
 

/etc/passwd:postgres:x:90:90:PostgreSQL Reserved UID:/:/usr/bin/pfksh
 

/etc/user_attr:postgres::::type=role;profiles=Postgres Administration,All
 

/etc/security/exec_attr:Postgres Administration:solaris:cmd:::/usr/postgres/8.2/bin/initdb:uid=postgres
/etc/security/exec_attr:Postgres Administration:solaris:cmd:::/usr/postgres/8.2/bin/ipcclean:uid=postgres
/etc/security/exec_attr:Postgres Administration:solaris:cmd:::/usr/postgres/8.2/bin/pg_controldata:uid=postgres
/etc/security/exec_attr:Postgres Administration:solaris:cmd:::/usr/postgres/8.2/bin/pg_ctl:uid=postgres
/etc/security/exec_attr:Postgres Administration:solaris:cmd:::/usr/postgres/8.2/bin/pg_resetxlog:uid=postgres
/etc/security/exec_attr:Postgres Administration:solaris:cmd:::/usr/postgres/8.2/bin/postgres:uid=postgres
/etc/security/exec_attr:Postgres Administration:solaris:cmd:::/usr/postgres/8.2/bin/postmaster:uid=postgres
With the permissions and dba account set up correctly, it should be ready to start.

Starting Postgres:
The Postgres database can be started from the  dba user.
V240/user$ su - root
Password:
V240/root$ su - postgres
V240/postgres$ svcadm enable svc:/application/database/postgresql_83:default_64bit

V240/postgres$ svcs "*postgres_83:default_64bit"
STATE    STIME   FMRI

offline* 0:43:27 svc:/application/database/postgresql_83:default_64bitsvcs: Pattern
 
V240/postgres$ svcs "*postgresql_83:default_64bit"
STATE   STIME   FMRI
online  0:43:37 svc:/application/database/postgresql_83:default_64bit
It may take a couple of minutes to start up for the first time, since many files from a sample database will need to be copied into the new directory structure, and onto the ZFS file systems.

Setting Up Sample Role/User, Database, and Client Access:
By default, all authenticated users are allowed to leverage the Postgres database under Solaris, but only on the same host. The default version of Postgres may be older than the version you wish to use.
V240/ivadmin$ type createdb psql
createdb is /usr/bin/createdb
psql is /usr/bin/psql


V240/ivadmin$ psql --version
psql (PostgreSQL) 8.1.18
contains support for command-line editing


V240/ivadmin$ createdb --version
createdb (PostgreSQL) 8.1.18
Since Solaris supports multiple versions of Postgres, it is important to set paths before using commands. A command should be used in the top of any script which runs Postgres or any command prompt where the user is intending on performing a lot of Postgres work.
V240/user$ PATH=/usr/postgres/8.3/bin:$PATH
V240/user$ export PATH
A privileged "role" (or "user") can set up a database and client access from another "user" or "role". The "createuser" binary is a wrapper around the "CREATE ROLE" command in Postgres.
In the case below, a new non-superuser (-S) will be created, which can create databases ("-d"), be restricted from creating new "roles" or "users" (-R), and log into the database ("-l"). Also, the binary command will echo the postgres command used ("-e"), for clarity sake.
V240/user$ su - root
Password:
V240/root$ su - postgres

V240/postgres$ PATH=/usr/postgres/8.3/bin:$PATH 
V240/postgres$ export PATH
V240/postgres$ createuser -S -d -R -l -e user
CREATE ROLE user NOSUPERUSER CREATEDB NOCREATEROLE INHERIT LOGIN;

(The addition of the proper path was used, in case it is not set up globally on the platform.)

The creation of the database can now be done by the Solaris user "user", which is also Postgres "role". By default, the name of the database is the same name as the "user".
V240/user$ PATH=/usr/postgres/8.3/bin:$PATH
V240/user$ export PATH
V240/ivadmin$ createdb -e
CREATE DATABASE user;

After the database is created, the
V240/ivadmin$ psql
Welcome to psql 8.3.8, the PostgreSQL interactive terminal.
Type:  \copyright for distribution terms
       \h for help with SQL commands
       \? for help with psql commands
       \g or terminate with semicolon to execute query
       \q to quit
user=>
The process of creating objects in the database can now take place.

Client Access Error:
If the user has never created a database, the first attempt access attempt will present an error such as:
V240/user$ psql
psql: FATAL:  database "user" does not exist

This indicates that a database must be created for that user.

Creating Database Error:
The "createdb" executable is a binary wrapper around the "create database" Postgres command. Databases are created by "cloning" a standard database template. If a database is created before the role is created, an error such as the following is presented:
V240/user$ createdb
createdb: could not connect to database postgres: FATAL:  role "user" does not exist
Before a database can be created, a user must be able to do this.

Creating Role Error:
A "role" is sometimes referred to as a "user". The Solaris user name is often tied directly as the "role". If the user is not privileged, the following error is presented:
V240/ivadmin$ createuser
Enter name of role to add: user
Shall the new role be a superuser? (y/n) n
Shall the new role be allowed to create databases? (y/n) n
Shall the new role be allowed to create more new roles? (y/n) n
createuser: could not connect to database postgres: FATAL:  role "user" does not exist
A privileged user must create new "roles". Under Solaris, this is the "postgres" user.

Resources:
Other Postgres resources are noted below:
[html] PostgreSQL 8.3.21 Documentation
[html] Dynamic Tracing of PostgreSQL via DTrace (in 8.3)
[pdf] Availability of PostgreSQL in the Data Center
[html] 2010-05 Setting up PostgreSQL under Solaris 10
[pdf] 2008-?? - Best Practices with PostgreSQL on Solaris
[html] 2005-11 - Tuning PostgreSQL under Solaris x64
[html] 2005-05 - Tuning Write Performance of PostgreSQL on Solaris
[html] 2005-04 - Tuning Solaris for PostgreSQL Read and Write Performance (8.0.2)

Monday, August 29, 2011

Technical Posts 2H August

Technical Posts 2H August

The following are technical articles related to Network Management in the past half-month.
  • Security: Devastating' Apache bug leaves servers exposed

    Attack code dubbed “Apache Killer” that exploits the vulnerability in the way Apache handles HTTP-based range requests was published Friday on the Full-disclosure mailing list. By sending servers running versions 1.3 and 2 of Apache multiple GET requests containing overlapping byte ranges, an attacker can consume all memory on a target system.

    The denial-of-service attack works by abusing the routine web clients use to download only certain parts, or byte ranges, of an HTTP document from an Apache server. By stacking an HTTP header with multiple ranges, an attacker can easily cause a system to malfunction.

  • Mobile: Dish eyes 4G LTE wireless network

    The radio spectrum owned by Dish, and LightSquared, is reserved for satellites, but as satellite transmissions have a hard time penetrating buildings and terrain operators are allowed to build an Ancillary Terrestrial Component* – infill transmitters operating at the same frequency as the birds and providing signal to those without line of sight.

    LightSquared turned that model on its head, suggesting that the ground-based network would be primary, with the satellite providing in-fill: estimated at around 2 per cent of traffic. LightSquared then successfully lobbied the FCC to permit it (and its wholesale customers) to ship equipment that isn't even capable of satellite communications, turning the company into a 4G network wholesaler without having to shell out for 4G spectrum.

  • Security: Worm spreading via RDP

    an Internet worm dubbed “Morto” spreading via the Windows Remote Desktop Protocol (RDP).

    F-Secure is reporting that the worm is behind a spike in traffic on Port 3389/TCP. Once it’s entered a network, the worm starts scanning for machines that have RDP enabled. Vulnerable machines get Morto copied to their local drives as a DLL, a.dll, which creates other files detailed in the F-Secure post.

    SANS, which noticed heavy growth in RDP scan traffic over the weekend, says the spike in traffic is a “key indicator” of a growing number of infected hosts. Both Windows servers and workstations are vulnerable.

  • Cloud: Java arrives on Heroku code cloud

    Heroku – the multi-language "platform cloud" owned by Saleforce.com – is now running Java applications.

    Akin to Google App Engine, Microsoft Azure, or VMware's Cloud Foundry, Heroku is an online service for building, deploying, and readily scaling applications. It was originally designed for Ruby on Rails apps, but has since expanded to Clojure, Node.js, and now Java.

  • Mobile: Why Apple is Removing Unique Identifiers

    Apple is planning to phase out unique device identifiers from iOS 5, according to documentation sent out to developers, possibly to stop people worrying about their privacy on iPhones and iPads... they should "create a unique identifier specific to your app".

    [Wall Stree Journal] Henschel also pointed to the recent spat between the notoriously secretive Apple and analytics firm Flurry as a possible spur for the move. In January, Flurry reported that it had identified around 50 tablet devices in testing at Apple's campus in Cupertino using its analytics.

    "Some company called Flurry had data on devices that we were using on our campus – new devices," Jobs said live at the D8 conference in New York. "They were getting this info by getting developers to put software in their apps that sent info back to this company! So we went through the roof. It's violating our privacy policies..."

  • Mobile: Nokia accidentally unveils OS it should have had in 2009

    Nokia is expected to unveil the a major refresh of its Symbian OS today, bringing it bang up to date with competitive phones from two years ago. Owners of more recent Symbian^3 models should be able to update their handsets eventually.

    Four new devices are expected to be unveiled – either today, or very shortly. The Belle update should keep loyalists happy for some time to come. Performance and usability appear to have been improved greatly.

  • Cloud: Performance Monitoring is Someone Else's Problem

    “Amazon and Google don’t have an army of service operatives monitoring their farms,” says Graeme Swan, a partner at consultancy Ernst & Young. “They basically smashed as much infrastructure as they possibly could into warehouses, and then just assumed that capacity would be there. Now, clients are telling them they want a premium service. They are worried that they have no way of monitoring it or tweaking it. So there is no premium service.”

    You can buy as much premium support as you like (although some question how well it works). Premium performance streams? Not so much.

  • Cloud: VMware turns shrink ray on open source dev cloud

    On Wednesday, the virtualization giant introduced Micro Cloud Foundry, a free downloadable version of its Cloud Foundry service that runs on a single laptop. This past spring, when VMware unveiled Cloud Foundry and open sourced the code behind it, the company indicated it would eventually offer a shrunken incarnation that would allow developers to test applications on their local machines.

  • Cloud: VMware turns self into (virtual) database co.

    vFabric Data Director has a utility pricing model, as you would expect, at a cost of $600 per VM under management per year that is running a database image. vFabric Postgres, VMware's tweaked and tuned version of PostgreSQL, is available free of charge for developers and will be available for download starting today at cloudfoundry.com.

    If you put a vFabric Postgres image into production, then it costs $1,700 per VM per year. The underlying vFabric 5.0 Standard Edition costs $1,200 per VM per year, while the Advanced Edition, which has more bells and whistles, costs $1,800 per VM. The Advanced Edition includes RabbitMQ messaging and an SQL interface for GemFire called SQLFire.

  • Cloud: Dell floats cloud built on ... VMware

    At the VMworld virtualization and cloud extravaganza in Las Vegas today, Dell said that it was fluffing up the Dell Cloud using VMware's brand-spanking-new ESXi 5.0 hypervisor, the vSphere 5.0 management tools for it, the vCloud Director cloud fabric, and the vCloud Connector extensions that allow a private cloud and a slice of a public cloud to be managed from the same console and to teleport jobs back and forth from the public and private clouds.

    The Dell Cloud comes out of the Dell Services unit, which is the amalgam of Dell's server and PC support business and consulting services practice with the Perot Systems system and application outsourcing business it acquired in September 2009 for $3.9bn.

  • Cloud: HP mates blades with VMware vSphere

    The VirtualSystem VS2 configuration for vSphere 5.0 moves to a bladed server and bladed Lefthand P4800 SAN arrays. The VS2 setup has a dozen BL460c G7 two-socket Xeon blade servers and two BladeSystem c7000 blade server chassis. Each blade has a dozen cores running at 3.06GHz

    The largest VirtualSystem for vSphere 5.0 setup is the VS3 box, which is designed to support up to 6,000 VMs. This monster has four BladeSystem c7000 chassis, a total of 64 of HP's ProLiant BL460c G7 servers

  • Mobile: Samsung 'considering purchasing' HP's orphaned webOS

    Samsung may be mulling over the purchase of webOS – recently orphaned by HP – in a move to protect itself from an increasingly unfriendly Apple and the threat of Google and its new toy, Motorola Mobility.

    Or so say "sources from notebook players", speaking with the Taiwanese rumor-and-news website, DigiTimes.

  • Cloud: VMware orders vCloud army across five continents

    VMware envisions a world where applications can roam across one big intercloud. Apps won't just jump from internal data centers to public cloud services, the company believes. They'll move from cloud to cloud like phone calls across cell networks.

    That's why VMware is keen on getting its vSphere server virtualization not only in the corporate data center, but out there on the service providers who want to be the next Amazon EC2. Then VMware can own the corporate cloud computing on both sides of the firewall.

  • Cloud: Citrix Cloud.com goes open source

    After taking control of the CloudStack cloud management framework through its acquisition of Cloud.com back in July, Citrix Systems is now open sourcing the code behind the tool. At the same time, it's adding support for the provisioning of workloads on additional hypervisors and, for the first time, on bare-metal machines.

    Cloud.com was founded in 2008 at about the same time as rival Eucalyptus Systems. It was known as VMOps before it came out of stealth mode in May 2010. Citrix is trotting out CloudStack 2.2.10, which has been certified to support rival VMware's ESXi 5.0 hypervisor, part of the vSphere 5.0 server virtualization stack that was annoumced in July and which started shipping last week.

    Citrix is trotting out CloudStack 2.2.10, which has been certified to support rival VMware's ESXi 5.0 hypervisor, part of the vSphere 5.0 server virtualization stack that was announced in July and which started shipping last week.

  • Internet: The case for a free market in IPv4 addresses

    Officially, the world ran out of IPv4 addresses earlier this year, when a final batch of addresses was divided among the five Regional Internet Registries. There are still a lot of unused and underused IP addresses in the hands of various private organizations. All that is needed is an incentive for them to part with their unused addresses voluntarily. In other words, what's needed is a market in IP addresses.

    Earlier this year, Microsoft paid $7.5 million for two-thirds of a million IP addresses that were previously held by a bankrupt Nortel, suggesting that the going rate for an IP address is around $10.

    Ford, Merck, Xerox, Halliburton, and nearly a dozen other companies not primarily in the networking business were each given a Class A block of 16 million addresses. MIT also got a Class A block, and the UK government got two of them. The US government claimed about a dozen Class A blocks, giving it control of nearly 200 million addresses—more IP addresses than all of Latin America has today.

  • Mobile: Sprint to get seat at grown-up table when iPhone 5 hits?

    Sprint will be the next carrier to offer the iPhone to customers in the US, according to sources speaking to the Wall Street Journal. The carrier will begin offering the iPhone 5 in October alongside AT&T and Verizon, both of which are also expected to begin selling the device mid-month, though it is believed that Sprint will also carry the iPhone 4, bolstering earlier rumors that Apple would keep around the iPhone 4 as the new low-cost replacement for the iPhone 3GS.

Network Management Connection

The transition from IP Version 6 from IP Version 4 may be a slow moving target. With companies like Microsoft buying large blocks and other companies holding millions - IP Addresses are like gold and oil. These investments may prove to not only be profitable, but the sale of these virtual goods may slow the implementation of IP V6.

Cloud Computing, based upon Virtualization technology from VMWare and Citrix open-sourced Xen continues to try to make inroads. Large system vendors like HP and Dell align themselves with proprietary VMWare. Oracle's VM technology maintains some level of compatibility with Citrix Xen. With Cloud Computing, the network becomes vastly more important.

Microsoft Windows has another WORM exposure, around it's proprietary RDP technology, based partially around file transfer options while UNIX Apache finds itself vulnerable to denial-of-service attacks. These key infrastructure points underpin modern intranet and internet computing today, both putting pressure on "the network". The Apache DoS merely makes it "look like" a network problem while another Microsoft worm really creates a possible capacity issue with "the network", if it can't be controlled.

HP finds itself bailing out of the mobile handset market (with Samsung possibly trying to buy it up for patent protections) dominated by heavy weights like Google (who ate Motorola's handset division and creates a mostly open-sourced based Android solution), Apple (with it's popular iPhone BSD UNIX parially open sourced solution), and Oracle (who is assaulting Google for using Java without paying license fees, like every other mobile vendor does.)

The mobile market has the opportunity to heat up, with more mobile 4G vendors hitting the market. By diversifying 4G to include Satellite vendors, in conjunction with Cell Phone operators, as well as land-line operators, in addition to new WiMax vendors (i.e. Clear) - there is the opportunity for a real explosion in the mobile network arena... which will all need to be managed. With dominant smart phone vendors like Apple possibly releasing iPhone for Sprint - this could really grow their market, as ATT and Verizon raise their costs to customers.