Showing posts with label component. Show all posts
Showing posts with label component. Show all posts

Monday, March 19, 2012

add total row to file via script component

Hi, I have a flat file source going into a script component which manipulates some values and puts those values in a flat file destination. What I now need to do is total a column within the script component and add that value as a row to the destination. At the moment data from source looks something like

ABC, 3

DEF, 5

FGH, 6

I have a variable in script component holding the running total I now want to add that to destination as a row e.g.

ABC, 3

DEF, 5

FGH, 6

Total, 14

This is probably very easy but I don't now how to add the Total row via the component. At the moment all rows are manipulated in the ProcessInputRow sub of component and output column values are assigned here. How do I add the Total row as I need to do this once last input row is complete.

Thanks

You'll need to use an asynchronous output to accomplish this. You could add an async output in addition to the synchronous output you get by default, and just write the running total to it.

See this topic in Books Online:

"Creating an Asynchronous Transformation with the Script Component"

|||You could also just use a multicast and an Aggregate transform to accomplish the same thing.|||Great thanks, Asynchronous Transformation worked

add total row to file via script component

Hi, I have a flat file source going into a script component which manipulates some values and puts those values in a flat file destination. What I now need to do is total a column within the script component and add that value as a row to the destination. At the moment data from source looks something like

ABC, 3

DEF, 5

FGH, 6

I have a variable in script component holding the running total I now want to add that to destination as a row e.g.

ABC, 3

DEF, 5

FGH, 6

Total, 14

This is probably very easy but I don't now how to add the Total row via the component. At the moment all rows are manipulated in the ProcessInputRow sub of component and output column values are assigned here. How do I add the Total row as I need to do this once last input row is complete.

Thanks

You'll need to use an asynchronous output to accomplish this. You could add an async output in addition to the synchronous output you get by default, and just write the running total to it.

See this topic in Books Online:

"Creating an Asynchronous Transformation with the Script Component"

|||You could also just use a multicast and an Aggregate transform to accomplish the same thing.|||Great thanks, Asynchronous Transformation worked

Thursday, March 8, 2012

Add replication component

Hello,
I have a publication that is pushed from a SQL 2005 Standard box to
about 40 SQL express subscribers. One of the servers went down and was
rebuild and I reran the same install script as I ran on the other
servers ( I made the script). I pushed out the same subscription as
before to it, and it is running smoothly (transactional). I then
needed to set up a snapshot subscription to all servers, and this
server was the only one that would not allow me to. When I try and set
up a new subscription to it I get this message:
"Replication components are not installed on this server. Run SQL
Server SETUP again and select the option to install replication.
(Microsoft SQL Server, Error: 21028).
So I thought I would just redo the setup and only pick the replication
component, but you cant, it makes you pick the whole database engine
tree to get it, and then the install says that it is already installed
and won't let you continue. Same happens if I go through add/remove
programs and use the add component link.
Any thoughts or ideas on this error. Right now the only solution I
have is reinstall from scratch.
OK, that error message was totally wrong. Since adding it through the
UI was not working, I added another subscription through the UI and at
the end generated a script file instead of actually finishing it. I
then changed the script file to point at my stubborn server that
supposedly did not have replication components, and the script ran fine
and the subscription was created, and worked.
PT
|||Can you try to create a local publication on this server? I am not sure if
this message is erroneous or not.
Hilary Cotter
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com
"Paul T." <weluvpaul@.hotmail.com> wrote in message
news:1168454660.767978.320110@.p59g2000hsd.googlegr oups.com...
> Hello,
> I have a publication that is pushed from a SQL 2005 Standard box to
> about 40 SQL express subscribers. One of the servers went down and was
> rebuild and I reran the same install script as I ran on the other
> servers ( I made the script). I pushed out the same subscription as
> before to it, and it is running smoothly (transactional). I then
> needed to set up a snapshot subscription to all servers, and this
> server was the only one that would not allow me to. When I try and set
> up a new subscription to it I get this message:
> "Replication components are not installed on this server. Run SQL
> Server SETUP again and select the option to install replication.
> (Microsoft SQL Server, Error: 21028).
> So I thought I would just redo the setup and only pick the replication
> component, but you cant, it makes you pick the whole database engine
> tree to get it, and then the install says that it is already installed
> and won't let you continue. Same happens if I go through add/remove
> programs and use the add component link.
> Any thoughts or ideas on this error. Right now the only solution I
> have is reinstall from scratch.
>

Saturday, February 25, 2012

add extra path to new component fails

I have a derived column component which has an regular output and an error output.

This works fine.

I want to add an extra regular output path.

I create a new OLEDB destination component and name it X

I right click on the derived column component and select "add path"

I then select X as the recipient of the new path.

And it errors with this garbage below

How can all the inputs be used up, it is a new component! It doesn't have any inputs!

Anyone know why this is happening (apart from the fact that it is not possible for me to go more than about 4 hours without encoutering another cryptic SSIS error that has no corresponding help topic on the MS website)

error message: (once again the MS help topic returns nothing)

===================================

All the available inputs on the target component are connected to outputs. Edit this component to define new inputs before proceeding. (Microsoft Visual Studio)


For help, click: http://go.microsoft.com/fwlink?ProdName=Microsoft%u00ae+Visual+Studio%u00ae+2005&ProdVer=8.0.50727.762&EvtSrc=Microsoft.DataTransformationServices.Design.SR&EvtID=DtsComponentNoOutputsFree&LinkId=20476


Program Location:

at Microsoft.DataTransformationServices.Design.PipelineTaskDesigner.AddNewPath(PipelineComponentMetadata sourceDtsComponentObject, PipelineComponentMetadata destinationDtsComponentObject)
at Microsoft.DataTransformationServices.Design.DataFlowDiagram.OnAddPath(MenuCommand sender, CommandHandlingArgs args)
at Microsoft.DataWarehouse.Design.CommandHandlingInfoMap.HandleCommand(MenuCommand menuCommand)
at Microsoft.DataWarehouse.Design.ComponentDiagram.InvokeCommand(MenuCommand menuCommand)
at Microsoft.DataTransformationServices.Design.DataFlowDiagram.InvokeCommand(MenuCommand menuCommand)
at Microsoft.DataWarehouse.Design.ComponentDiagram.Microsoft.DataWarehouse.Interfaces.ICommandTarget.InvokeCommand(MenuCommand menuCommand)

Most transformations have only one output. If you would like for the same output information to go to more than one component you should use the multicast transform after the derived column and have that pass on the output to as many places as you would like.|||

how stupid that it is shown as a menu option, and that it even goes through the process of allowing you to select another component

|||

Adolf,

'Add Path' option is just the equivalent of joining DF components via red/green arrows; it is not for creating additional outputs in a component.

If you need an additional data stream, you would need to use the multicast transformation or perhaps an scritp component.

Add Extra Error Columns - From The Component?

I've read a MSDN article on enhancing error outputs with a script component. This solution added the script component astride the error output, and it was able to add a column with the error description text.

I would like to do something a little different. I'd like to add information from my custom component. When the component causes an error, it generally "knows why" - it has available to it the information that made it decide there was an error. I'd like to include that information in error outputs, but I haven't seen a way to do that.

Is this even possible using the error output directly? I could abandon error outputs and use Conditional Split, along with an "iserror" column plus my extra columns, but I'd prefer not to do that.

This might be possible. Try the following (I'm assuming your custom component is synchronous):

1. In your implementation of ProvideComponentProperties, create 2 outputs that are synchronous to the same input, put them in the same exclusion group and mark one of them as the error output. By marking it as an error output, you'll automatically get the 2 error output columns and the output should show up red in the designer. Then add a third column to that output for your error description.

2. In PreExecute, use FindColumnByLineageID to get the handle to the error description column you added. If you can't get the handle, then this isn't possible (I haven't tried it, but I can't think of anything in the API to keep you from getting it).

3. If you got the handle, use it in ProcessInput to set the error description before calling RedirectErrorRow.

|||

Ted Lee - MSFT wrote:

This might be possible. Try the following (I'm assuming your custom component is synchronous):

1. In your implementation of ProvideComponentProperties, create 2 outputs that are synchronous to the same input, put them in the same exclusion group and mark one of them as the error output. By marking it as an error output, you'll automatically get the 2 error output columns and the output should show up red in the designer. Then add a third column to that output for your error description.

That part worked. The extra column was added to the error output. Unfortunately:

Ted Lee - MSFT wrote:

2. In PreExecute, use FindColumnByLineageID to get the handle to the error description column you added. If you can't get the handle, then this isn't possible (I haven't tried it, but I can't think of anything in the API to keep you from getting it).

The above caused an error during PreExecute:

The output "TranslateProvidersErrorOutput" (347) is a synchronous output and the buffer type cannot be retrieved for a synchronous output.

The code I tried is:

Code Snippet

public override void ProvideComponentProperties()

{

ComponentMetaData.UsesDispositions = true;

IDTSInput90 input = ComponentMetaData.InputCollection.New();

input.Name = "TranslateProvidersInput";

input.ErrorRowDisposition = DTSRowDisposition.RD_FailComponent;

IDTSOutput90 output = ComponentMetaData.OutputCollection.New();

output.Name = "TranslateProvidersOutput";

output.SynchronousInputID = input.ID;

output.ExclusionGroup = 1;

// The new code:

IDTSOutput90 errorOutput = ComponentMetaData.OutputCollection.New();

errorOutput.Name = "TranslateProvidersErrorOutput";

errorOutput.SynchronousInputID = input.ID;

errorOutput.ExclusionGroup = 1;

errorOutput.IsErrorOut = true;

IDTSOutputColumn90 extraColumn = errorOutput.OutputColumnCollection.New();

extraColumn.Name = "ErrorDescription";

extraColumn.SetDataTypeProperties(DataType.DT_WSTR, 255, 0, 0, 0);

_extraColumnLineageID = extraColumn.LineageID;

// The original code:

//AddErrorOutput(

//"TranslateProvidersErrorOutput", input.ID, output.ExclusionGroup);

IDTSRuntimeConnection90 connection =

ComponentMetaData.RuntimeConnectionCollection.New();

connection.Name = "ADONET";

}

public override void PreExecute()

{

IDTSOutput90 errorOutput = ComponentMetaData.OutputCollection[1];

_extraColumnIndex =

BufferManager.FindColumnByLineageID(

errorOutput.Buffer, _extraColumnLineageID);

}

The hightlighted code is what failed.

I don't yet understand how buffers work with multiple outputs, so I can't figure out any other way to get the column index. I'm stuck at that point, for now.

If anyone has more ideas, I'll be glad to try them. One workaround I thought of was to add the extra columns to the default output and see if they get replicated to the error output when the default error output is created. This would have the unfortunate side-effect of proliferating error-only columns throughout the data flow, but should work well enough if the extra columns are truly required.

It will be necessary for components in the flow to not add one of these columns if it is already present.

|||

John,

I've got this working in my destination component: adding error messages, row #, time, etc.

The difference I think is in this line:

IDTSOutput90 errorOutput = ComponentMetaData.OutputCollection[1];

_extraColumnIndex =

BufferManager.FindColumnByLineageID(

errorOutput.Buffer, _extraColumnLineageID);

The equivalent in my code would be:

IDTSInput90 input = ComponentMetaData.InputCollection[0];

_extraColumnIndex =

BufferManager.FindColumnByLineageID(

input.Buffer, _extraColumnLineageID);

It is the input buffer that actually has the slots for the new error columns.

Martin

Add Extra Error Columns - From The Component?

I've read a MSDN article on enhancing error outputs with a script component. This solution added the script component astride the error output, and it was able to add a column with the error description text.

I would like to do something a little different. I'd like to add information from my custom component. When the component causes an error, it generally "knows why" - it has available to it the information that made it decide there was an error. I'd like to include that information in error outputs, but I haven't seen a way to do that.

Is this even possible using the error output directly? I could abandon error outputs and use Conditional Split, along with an "iserror" column plus my extra columns, but I'd prefer not to do that.

This might be possible. Try the following (I'm assuming your custom component is synchronous):

1. In your implementation of ProvideComponentProperties, create 2 outputs that are synchronous to the same input, put them in the same exclusion group and mark one of them as the error output. By marking it as an error output, you'll automatically get the 2 error output columns and the output should show up red in the designer. Then add a third column to that output for your error description.

2. In PreExecute, use FindColumnByLineageID to get the handle to the error description column you added. If you can't get the handle, then this isn't possible (I haven't tried it, but I can't think of anything in the API to keep you from getting it).

3. If you got the handle, use it in ProcessInput to set the error description before calling RedirectErrorRow.

|||

Ted Lee - MSFT wrote:

This might be possible. Try the following (I'm assuming your custom component is synchronous):

1. In your implementation of ProvideComponentProperties, create 2 outputs that are synchronous to the same input, put them in the same exclusion group and mark one of them as the error output. By marking it as an error output, you'll automatically get the 2 error output columns and the output should show up red in the designer. Then add a third column to that output for your error description.

That part worked. The extra column was added to the error output. Unfortunately:

Ted Lee - MSFT wrote:

2. In PreExecute, use FindColumnByLineageID to get the handle to the error description column you added. If you can't get the handle, then this isn't possible (I haven't tried it, but I can't think of anything in the API to keep you from getting it).

The above caused an error during PreExecute:

The output "TranslateProvidersErrorOutput" (347) is a synchronous output and the buffer type cannot be retrieved for a synchronous output.

The code I tried is:

Code Snippet

public override void ProvideComponentProperties()

{

ComponentMetaData.UsesDispositions = true;

IDTSInput90 input = ComponentMetaData.InputCollection.New();

input.Name = "TranslateProvidersInput";

input.ErrorRowDisposition = DTSRowDisposition.RD_FailComponent;

IDTSOutput90 output = ComponentMetaData.OutputCollection.New();

output.Name = "TranslateProvidersOutput";

output.SynchronousInputID = input.ID;

output.ExclusionGroup = 1;

// The new code:

IDTSOutput90 errorOutput = ComponentMetaData.OutputCollection.New();

errorOutput.Name = "TranslateProvidersErrorOutput";

errorOutput.SynchronousInputID = input.ID;

errorOutput.ExclusionGroup = 1;

errorOutput.IsErrorOut = true;

IDTSOutputColumn90 extraColumn = errorOutput.OutputColumnCollection.New();

extraColumn.Name = "ErrorDescription";

extraColumn.SetDataTypeProperties(DataType.DT_WSTR, 255, 0, 0, 0);

_extraColumnLineageID = extraColumn.LineageID;

// The original code:

//AddErrorOutput(

//"TranslateProvidersErrorOutput", input.ID, output.ExclusionGroup);

IDTSRuntimeConnection90 connection =

ComponentMetaData.RuntimeConnectionCollection.New();

connection.Name = "ADONET";

}

public override void PreExecute()

{

IDTSOutput90 errorOutput = ComponentMetaData.OutputCollection[1];

_extraColumnIndex =

BufferManager.FindColumnByLineageID(

errorOutput.Buffer, _extraColumnLineageID);

}

The hightlighted code is what failed.

I don't yet understand how buffers work with multiple outputs, so I can't figure out any other way to get the column index. I'm stuck at that point, for now.

If anyone has more ideas, I'll be glad to try them. One workaround I thought of was to add the extra columns to the default output and see if they get replicated to the error output when the default error output is created. This would have the unfortunate side-effect of proliferating error-only columns throughout the data flow, but should work well enough if the extra columns are truly required.

It will be necessary for components in the flow to not add one of these columns if it is already present.

|||

John,

I've got this working in my destination component: adding error messages, row #, time, etc.

The difference I think is in this line:

IDTSOutput90 errorOutput = ComponentMetaData.OutputCollection[1];

_extraColumnIndex =

BufferManager.FindColumnByLineageID(

errorOutput.Buffer, _extraColumnLineageID);

The equivalent in my code would be:

IDTSInput90 input = ComponentMetaData.InputCollection[0];

_extraColumnIndex =

BufferManager.FindColumnByLineageID(

input.Buffer, _extraColumnLineageID);

It is the input buffer that actually has the slots for the new error columns.

Martin

Thursday, February 16, 2012

Add a variable from custom component

Hi

I am writing a custom transformation component that utilises a user variable.

Before using the variable at run time I am checking that the variable exists, but it would be nice to be able to add it if it does not. I cannot find any documentation on the subject, though I can see that the Variables class is derived from a ReadOnlyCollectionBase.

Is there a way to add a user variable with package scope from a custom component, either at run time or design time?

Thanks . . . Ed

I do not think it is possible currently.

Thanks.

Sunday, February 12, 2012

Ad Hoc Distributed Queries Error

I am getting the following error when with SQL Express.

SQL Server blocked access to STATEMENT 'OpenRowset/OpenDatasource' of component 'Ad Hoc Distributed Queries' because this component is turned off as part of the security configuration for this server. A system administrator can enable the use of 'Ad Hoc Distributed Queries' by using sp_configure. For more information about enabling 'Ad Hoc Distributed Queries', see "Surface Area Configuration" in SQL Server Books Online.

While I was very please to see such a verbose error and directions on where to find the answer I have yet to figure out how to turn this option on...

I tried sp_configure'Ad Hoc Distributed Queries', 1 but got the following error

The configuration option 'Ad Hoc Distributed Queries' does not exist, or it may be an advanced option.

If I execute only sp_configure it does not list Ad Hoc Distributied Queries as an option. I checked the sql Books Online and it tells me to use the Surface configuration tool which SQL Express does not seem to have...

Could someone help me out with this?

Thanks - Mark

As 'Ad Hoc Distributed Queries' is an advanced option, you need to turn on advanced options. Try the statements below:

sp_configure 'show advanced options',1
reconfigure with override
go
sp_configure 'Ad Hoc Distributed Queries',1
reconfigure with override
go

|||

FANTASTIC! Quick and easy!

That works - Thanks